A Delphi study to identify indicators of poorly managed pain for pediatric postoperative and procedural pain

1London South Bank University, London, United Kingdom; 2IWK Health Centre; 3Dalhousie University; 4Capital District Health Authority, Halifax, Nova Scotia; 5Centre for Neuroscience, University of Alberta, Edmonton, Alberta Correspondence: Dr Alison Twycross, Faculty of Health and Social Care, K2 Building, London South Bank University, 103 Borough Road, London SE1 0AA, United Kingdom. Telephone 44-0-20-7815-8419, e-mail twycrosa@lsbu.ac.uk Health care systems are increasingly focusing on preventing adverse events. An adverse event has been defined as: An injury related to medical management, in contrast to complications of disease. Medical management includes all aspects of care, including diagnosis and treatment, failure to diagnose or treat, and the systems and equipment used to deliver care. Adverse events may be preventable or nonpreventable (1).

Pain Res Manag Vol 18 No 5 September/October 2013 e69 The importance of preventing adverse events, such as hospitalacquired infections, has received increasing attention in the past five to 10 years and, consequently, the quality of care has improved (11).Research regarding quality improvement (QI) has been advanced by developing tools to identify when adverse events have occurred (12) (eg, trigger factors or indicators) and procedures to examine the contributors to these events (eg, root-cause analysis).It has previously been argued that children's pain management in hospital may be improved if we apply a similar adverse events framework to the treatment of pain (5).To study whether treating pain as an adverse event can improve practice, we need to develop tools that will assist in the identification of undermanaged pain in children in hospital.As a first step in this process, the present study aimed to develop a set of indicators that may identify when children's acute pain has been undermanaged in hospital, using expert consensus.

MeThoDs
The Delphi technique was developed in the 1950s as a means of scientific and technological forecasting (13).It relies on the judgment of an expert panel and aims to develop consensus about a given subject area (14).When undertaking a Delphi study, a group of experts in the field are identified and asked about their opinion on the question of interest.The first round of a Delphi study is used to generate ideas, which are reconsidered and rated by experts in subsequent rounds (15).An outline of the present study is provided in Figure 1.

expert panel
The sample consisted of experts in two fields: pediatric pain management and QI.For the purposes of the present study, an expert in QI was defined as a member of the Canadian Association of Pediatric Health Centres National Patient Safety Collaborative (NPSC).At the time of data collection, there were 46 members in the NPSC.An expert in pediatric pain was defined as someone who has had: • Previous involvement in clinical research on this topic -defined as first author of at least one article or second author of two peerreviewed articles relating to pediatric acute pain management; or • Previous involvement in pain clinical work -defined as at least two years experience working in a pediatric acute pain service.Only one of these criteria needed to be met because the intention was to include both researchers and clinicians.The goal was to recruit a multidisciplinary panel of ≥30 professionals for the study.
The research team -an international, multidisciplinary group with a combined expertise in pediatric pain management of >50 years and with >300 publications -developed a list of potential participants.PubMed was used to identify authors of articles with a clinical focus in the past five years using the search terms "pediatric pain management" and "pediatric pain assessment".Authors of articles known to the researchers were also included.Potential pediatric pain clinicians were identified in several ways including: respondents to the pediatric pain e-mail list over a six-week period whose e-mail signature indicated they had a predominately clinical role; and clinicians known to the research team.

Round 1: Data collection tools
A questionnaire was developed for the first round of the study and was administered using Opinio (a secure software for Internet data collection).Participants were asked: In your opinion, what indicators would signify that acute pain in a child has not been adequately controlled?They were provided with an example of three adverse event indicators.These examples related to areas of pain management it was anticipated that all participants would include in their list of indicators.These included: • A pain score was not documented; and • A procedure generally considered to be painful was completed without the use of analgesic/anesthetic being documented.It was emphasized that these indicators did not necessarily mean pain was undermanaged but were cues to further evaluate the adequacy of children's pain management.Examples were provided because the authors expected these would help experts to provide indicators.The only difference between the questionnaires for the two groups of experts was that pediatric pain experts were asked to provide some demographic data to ensure they met the criteria set for experts in the present study.

Round 1: Procedure
Approval was obtained from the Ethics Review Board at the IWK Health Centre (Halifax, Nova Scotia).Potential members of the pain expert panel were contacted individually by e-mail to request participation.The e-mail included an introduction letter and a link to the survey.Potential members of the QI expert panel were contacted by an e-mail sent from the Canadian Association of Pediatric Health Centres NPSC.This e-mail contained the same introductory letter and a link to the survey.Participants were asked to respond within a two-week period (15).A reminder e-mail was sent out at the end of this period and participants were allowed an additional week to return the questionnaire.

Round 1: Data analysis
The first author of the present study has extensive experience analyzing qualitative data and, therefore, took the lead for this part of the study.The responses provided by both sets of experts were grouped together in semantically similar themes (15) using the approached advocated by Creswell (16).The grouped data were read several times by the first author to identify recurrent responses that could be placed into themes or categories (17).This was performed manually by highlighting the text according to themes, using different colours of highlighter pens, cutting, pasting and collating the data.As is standard practice when undertaking qualitative data analysis, the responses were also reviewed and coded by another member of the research team to minimize researcher bias (18).The themes identified by the second coder were similar to those identified by the principal researcher but were worded slightly differently.After discussion between the researcher and the second coder, the final wording of the themes was agreed.This allowed a list of possible adverse event indicators to be produced.Given the small number of QI experts responding to the first round, data from the two groups were analyzed together.

Round 2: Data collection tool
An electronic questionnaire (e-questionnaire) was developed for this round of the study, using the indicators generated in data analysis in round 1. Round 2: Procedure The e-questionnaire was sent out to subjects who participated in the first round of the study.Respondents were asked to rate the importance of each indicator using a five-point Likert scale (ranging from 'very unimportant' to 'very important').The aim of this round was to establish a level of consensus regarding the relevant indicators in this context; therefore, participants were asked: "For each indicator please put an X in the box that you feel best describes the importance of the indicator of poor pain management practice."As in round 1, participants were asked to respond within a two-week period.A reminder e-mail was again sent out at the end of this period, and participants were allowed an additional week to return the questionnaire.

Round 2: Data analysis
Frequency tables for the items in round 2 were used to determine which of the potential adverse event indicators reached a 70% level of consensus.The level of consensus for each indicator was calculated by adding the frequencies for the 'important'/'very important' responses.

Round 1
A total of 122 pain experts were sent an e-mail asking them to consider participating in the study.Fifty-nine professionals returned responses in round 1.Two e-mails were returned, yielding a response rate of 49%.Demographic details for the pain experts are provided in Table 1.Not all participants provided responses to each demographic question and 57 provided data about what they considered adverse event indicators to be.Four QI experts provided responses in this round, representing a 9% response rate.
A list of possible adverse event indicators was derived from the round 1 data.The number of responses for each indicator are presented in Tables 2 and 3.During analysis, it became clear that some responses related to postoperative pain and others to procedural pain.A decision was made to develop a set of adverse event indicators for each type of pain.Indicators were further divided into those that could be found via a chart audit and those requiring observation or interaction with patients and families (termed 'observational' from here forward).Indicators were categorized as one or the other.Consistent with the  Pain medications prescribed on a regular basis but given less frequently than prescribed (CRI)

Documentation issues
There is no documentation in the child's notes about their pain or pain management (CRI)

(27.9)
There is no documentation about which pain assessment tool was used (CRI) PRN pain medication is given without a pain score being documented (CRI) Pain assessment undertaken but not recorded in the child's charts (OI)

Involvement in decision making
Child not involved in decision making about their pain management (if child is old enough) (OI) 4 (6.6) Parent not involved in decision making about their child's pain management (OI) 6 (9.8)

Child satisfaction
Child does not feel they were sufficiently involved in decision making about their pain management (if child is old enough) (OI) 3 (4.9) Child is not satisfied with the information they have been given about their pain management (OI) Child is not satisfied with pain care provided (OI) 6 (9.8)

Parent satisfaction
Parent does not feel they were sufficiently involved in decision making about their child's pain management (OI) 3 (4.9) Parent is not satisfied with pain care provided (OI) 13 (21.3)Parent is not satisfied with the information they have been given about their pain management (OI) Pain Res Manag Vol 18 No 5 September/October 2013 e71 Delphi technique (15), all of the potential indicators were included in round 2, even if they were only identified by a small number of experts.

Round 2
Forty-two experts provided responses in round 2. The level of consensus was calculated by adding the percentages of respondents indicating each indicator was 'important'/'very important'.The only indicator that did not achieve ≥70% consensus was: "No evidence that the child life/play therapist was involved in preparing the child for the procedure".In light of this result, a third round was not performed.Items achieving a consensus level ≥90% are shown in Tables 4 and 5.

DisCussion
The present two-round Delphi study resulted in a list of adverse event indicators for postoperative and procedural pain in children that can be identified by chart review or by observation of care and/or interaction with the child or family.Given the nature of pain and the evidence of lack of documentation of pain in some contexts, we believe a combination of chart review and prospective recording (observational indictors) will lead to the most valid estimates of this adverse event.
The high level of consensus obtained in round 2 suggests that the expert participants considered these indicators to be valid markers of the possibility that pain has been undermanaged.The indicators will be discussed in the context of previous research and the definition of an adverse event provided earlier.

Postoperative indicators
The postoperative indicators include six items on pain assessment.These relate to whether a developmentally appropriate pain assessment tool was used, the action taken if a pain score was ≥4 of 10, and whether pain medications are given without a pain score being documented.Other studies have found that pain assessments are not performed consistently or always recorded (4,8).Previous research has also found that even when pain scores are recorded they may not be used to guide decision making (19).Assessing a child's pain using a developmentally appropriate tool has been identified as best practice (20) and, if performed, may facilitate decision making regarding pain management.Four of the postoperative indicators relate to the reassessment of pain.Current best practice guidelines suggest that pain should be reassessed following the implementation of pain-relieving strategies (20).However, in practice, reassessment of pain does not always occur (21,22).Agreement is needed on the level of pain intensity at which action should be taken to relieve children's pain.Children have indicated their pain threshold for treatment postoperatively is 4.72 (on a scale of zero to 10) (6).Other studies have reported similar findings (7,23).In the present study, based on the results from round 1, a decision was made that failing to treat if a child reported a pain score ≥4 of 10 would be considered to be an adverse event.The level of consensus achieved for the two indicators relating to whether action was taken when a child reported a pain score ≥4 in round 2 (79.07%and 76.19%) suggests the experts believed that this was an appropriate choice.However, pain is a subjective phenomenon and individual pain thresholds vary (7,23).Children's experience of pain and behaviour when in pain are influenced by many factors such as temperament (24,25) and previous experiences of pain (26).This is clearly illustrated in one study, in which variability was found in the pain scores (on an 11-point numerical scale) at which children reported a perceived need for medicine (27).Identifying a definitive cut-off point for when pain should be treated may, therefore, be difficult.
Four of the postoperative indicators relate to administering pain medications.Previous research suggests the administration of analgesic drugs postoperatively varies in different settings (4,28).This is despite clinical guidelines clearly stating that an essential component of postoperative care is prescribing and administering analgesic drugs (2).Three indicators relate to the use of nonpharmacological methods postoperatively; two of these relate to documentation, and one to the reassessment of pain.Despite growing evidence that nonpharmacological methods enhance the effectiveness of analgesic drugs postoperatively (29), these strategies are not always used as often as they could be (21).Three of the chart review indicators and an observational indicator relate to documentation.Other studies have found that documentation relating to pain is not as thorough as it could be, with pain assessments not always documented, and pain-relieving interventions and their effectiveness not recorded in the child's notes (3,22).
One of the observational indicators for postoperative pain relates to involving children in decisions related to their pain management.The United Nations Convention on the Rights of the Child makes it clear that children have a right to be involved in decisions regarding their care (30).Current best-practice guidelines state that children should be involved in decision making related to their postoperative and procedural pain (2).Previous research indicates that there is minimal verbal communication between nurses and children related to postoperative pain management, and that this relates to pain medications they plan to administer (21).Indeed, children have indicated that nurses do not discuss pain management with them as often as they would like (31).
Another postoperative observational indicator relates to the involvement of parents in decisions regarding their child's postoperative pain management.This is consistent with current best practice (2).Earlier research has found that nurses do discuss children's pain management with parents, but this is usually initiated by the parents, and involves nurses telling them about the analgesic drugs they will be administering No evidence that the child life/play therapist was involved in preparing the child for the procedure (CRI) 1 (1.6)No evidence that the parent was prepared for their role in supporting their child through the procedure (CRI) No analgesic drugs given before a painful procedure (CRI) 19 (31.1)No sucrose given before a painful procedure for a neonate (CRI) No evidence that a local anesthetic cream or vapu-coolant was used before venipuncture or cannulation (CRI) 3 (4.9) Child is distressed before the procedure (CRI) 1 (1.6)

Issues during the procedure
Child is distressed during the procedure (CRI) 1 (1.6)

Issues after the procedure
No pain plan documented (CRI) 2 (3.3)No documentation about which nonpharmacological methods were used to help the child cope with the painful procedure (CRI) 9 (14.8) Child is distressed after the procedure (CRI) The nonpharmacological methods used to help the child cope with the painful procedure are not documented (CRI) 9 (14.8)

Child satisfaction
Child is not satisfied with the management of pain during the procedure (OI)

Parent satisfaction
Parents did not feel prepared for their role in supporting the child through the procedure (OI) Parents are not satisfied with the management of pain during the procedure (OI) 13 (21.3)Pain Res Manag Vol 18 No 5 September/October 2013 e72 (21).Evidence of considerable variability in the amount, content and clarity of information given to parents regarding their child's postoperative pain management has been found (32).Parents have also indicated that they would like more information about their child's pain management (33,34).

Procedural pain
Preparing children for painful procedures helps them better cope with the experience (35) and has been identified as best practice (36).Parents also need preparation for their role in supporting children through a painful procedure (37); playing an active role during the procedure may help reduce their sense of helplessness and anxieties (38).In the present study, experts identified lack of preparation of parents and children as potential indicators of under managed pain.Unfortunately, adequate preparation does not always happen (39).Three observational indicators relate to a child being distressed before, during or after a procedure.Painful procedures can cause a large amount of distress for children and their families (2).However, if the current best practice guidelines in this area (2,36) are followed, this distress and anxiety should be minimized.Individual children's behaviour before, during or after a painful procedure varies and is affected by several factors including age (40) and culture (41), as well as previous experiences of pain (42).During the next stage of testing, careful consideration needs to be given to individual variability in children's distress response to painful procedures and the role of health care providers in minimizing this response.The wording of the indicators relating to children's distress before, during and after procedures may need amending, or they may be removed completely.
The use of local anesthetic creams and analgesic drugs during painful procedures is considered to be best practice (2,36).Three of the indicators relate to administering analgesic drugs and local anesthetics before painful procedures.However, these are not always used in practice, as demonstrated by the results of a chart audit undertaken across eight children's hospitals in Canada (43).Three indicators relate to the use of nonpharmacological methods for procedural pain.Despite growing evidence that nonpharmacological methods help children cope with painful procedures (44), these strategies are not always used as often as they could be (21).Sucrose is technically a drug.However, its administration to neonates before painful procedure is usually considered a nonpharmacological method and is also known to diminish pain behaviours in neonates (45).Best-practice guidelines indicate that a pain plan should be produced and documented for all children undergoing painful procedures (36).However, in reality, there is often little documentation relating to the interventions used to manage children's painful procedures (43).This may contribute to the inadequate treatment of pain (46).

satisfaction with care
The observational indicators for both postoperative and procedural pain include items relating to child and parental satisfaction with the pain care provided.It is noteworthy that in previous studies, children and parents have indicated that they are satisfied with the pain management provided, even if they experienced moderate to severe pain (4,47).Care, therefore, needs to be exercised when drawing conclusions about practice from these indicators.However, it could be argued that if parents and children are usually satisfied with pain care, a response indicating that they are not satisfied has greater importance and should be explored in the same way as an adverse event.

The indicators compared with the definition of an adverse event
The definition provided at the beginning of the present article makes it clear that failure to diagnose or treat should be considered to be an adverse event (1) and, as such, relates to the indicators produced in the present study.Not assessing or reassessing pain constitutes a failure to diagnose.Failing to assess or reassess pain may also result in a failure to treat.Not administering or prescribing sufficient analgesic drugs postoperatively or before a painful procedure can also be considered to be a failure to treat, as can not using nonpharmacological methods to help manage postoperative

Reassessment issues
There is no evaluation of the effectiveness of the pain-relieving interventions used No reassessment of pain documented following the administration of around-the-clock pain medication Pain intensity is not reassessed following the administration of PRN medications No reassessment of pain documented following the implementation of a nonpharmacological method of pain relief

Analgesic drug issues
No analgesic drugs are prescribed The doses of analgesic drugs prescribed are at a subtherapeutic level

Documentation issues
There is no documentation about which pain assessment tool was used There is no documentation in the child's notes about their pain or pain management

Involvement in decision making
Child not involved in decision making about their pain management (if child is old enough) Parent not involved in decision making about their child's pain management

Child satisfaction
Child is not satisfied with the information they have been given about their pain management Child is not satisfied with pain care provided Child does not feel they were involved sufficiently in decision making about their pain management (if child is old enough)

Parent satisfaction
Parent is not satisfied with pain care provided Parent is not satisfied with the information they have been given about their pain management Parent does not feel they were involved sufficiently in decision making about their child's pain management PRN Pro re nata (as required)

Issues before the procedure
No analgesic drugs given before a painful procedure Parents not prepared for their role in supporting their child through the procedure Child is not prepared for the procedure Child is distressed before the procedure

Issues during the procedure
Child is distressed during the procedure

Issues after the procedure
Child is distressed after the procedure

Child satisfaction
Child is not satisfied with the management of pain during the procedure

Parent satisfaction
Parents did not feel prepared for their role in supporting the child through the procedure Parents are not satisfied with the management of pain during the procedure Pain Res Manag Vol 18 No 5 September/October 2013 e73 or procedural pain.Not preparing a child and their parents for a painful procedure or a child demonstrating significant distress throughout the procedure also constitutes a failure to treat.A lack of documentation may contribute to the inadequate treatment of pain (46) and, therefore, also fits the definition of an adverse event.Not involving children in decision making is a breech of their human rights (30) and can be considered as a failure to deliver ethically sound care.It also means that current best practice guidelines were not followed, thus constituting a failure to treat.This also applies to not involving parents in decision making.The indicators developed in the present study clearly relate to the definition of an adverse event.This supports the contention, proposed at the beginning of the present article, that mismanaged and undertreated pain due to procedures or surgery should be considered to be an adverse event.

other considerations
If indicators, such as those developed in the present study, are to be applied more widely in clinical practice, it will be necessary for staff in QI and pain management to work together.There was limited involvement of QI experts in the present study.This may be due, at least in part, to the fact they were approached via a third party rather than by the researchers directly.QI experts may need additional educational input about the consequences of unrelieved pain and, thus, the importance of better pain management standards in hospitals.Pain management experts may need training in QI techniques and in health services research to understand the types of data used by health care managers to make policy decisions.The successful implementation of these indicators into practice, following further testing, will necessitate consideration of these factors.

Limitations of study
A few limitations to the present study should be noted.In addition to the relatively low response rate of QI experts, the identification and response rate of pain experts may also have been a limitation.Although our actual recruitment (n=42) exceeded our original aim of 30 respondents, this still represents a response rate of <50%.Because only 120 pain experts were asked to consider taking part in the study, it is possible the views of those not included would differ from those who participated.
However, the high level of consensus achieved in round 2 suggests there is a level of agreement about what constitutes mismanaged pain.Furthermore, participants were not evenly divided between professional groups (Table 1), with nearly 60% being nurses; this may have impacted the responses received.However, the breakdown between the professional groups is believed to be representative of those working in pain management.Finally, it is possible that by providing sample indicators, we may have biased participants' responses.We provided these indicators in an attempt to clarify our question, but we acknowledge that this may have led respondents to provide only similar indicators.

ConCLusion
The adverse care indicators developed in the present study are the first step to conceptualizing mismanaged pain as an adverse event, but require further refinement.There is a need to test their usability in the clinical setting and to determine whether a smaller number of indicators will serve the intended purpose.Further research also needs to be conducted to determine how these indicators are best identified (ie, either via a chart review or through observation).Perhaps most importantly, there is a need to determine whether the indicators developed identify potential adverse events in clinical practice.However, the indicators developed are an important first step in moving to a situation in which mismanaged pain is considered to be an adverse event.
suMMARy Significant pain is often caused by pediatric health care; unrelieved moderate or severe pain in this context could be considered to be an adverse event.One strategy for identifying adverse events is using lists of indicators.The Delphi technique was used to survey experts in pediatric pain management and QI to identify indicators that could be used to detect cases in which acute pain was undermanaged.A set of chart review and observational indicators for undermanaged pediatric postoperative and procedural pain have been developed.

FunDinG:
The first author undertook this research while on an inter- national research sabbatical funded by the Faculty of Health and Social Care Sciences, Kingston University and St George's University of London.

Figure 1 ) 2 : 2 1 :
Figure 1) Outline of study.CAPCH Canadian Association of Pediatric Health Centres; NPSC National Patient Safety Collaborative; QI Quality improvement Responses received from 42 experts.Analysis of consensus.List of prospective and retrospective indicators developed.Responses received from 42 experts