How well does the Internet answer patients ’ questions about inflammatory bowel disease ?

1IBD Clinical and Research Centre, and Department of Clinical Health; 2Department of Psychology; 3Department of Internal Medicine, University of Manitoba, Winnipeg, Manitoba Correspondence: Dr Charles N Bernstein, Department of Internal Medicine, University of Manitoba, 804F-715 McDermot Avenue, Winnipeg, Manitoba R3E3P4.Telephone 204-789-3369, fax 204-789-3972, e-mail cbernst@cc.umanitoba.ca Received for publication April 8, 2010. Accepted June 1, 2010 The Internet is a resource that is increasingly accessible, fast and comprehensive for a variety of queries. People rely on the Internet as a primary source of information for many items, including their health. Among adults in the United States (US) and Europe, 61% and 52%, respectively, have searched online for health information (1,2). Furthermore, information acquired from the Internet may influence how patients manage their health themselves or with the aid of their doctors (3,4). Fifty-nine per cent of patients recently diagnosed with an illness reported that the information they found online led them to ask a doctor new questions or to get a second opinion (4). As the trend away from paternalistic medicine toward more patient-centred practice continues, it is clear that information patients access through the Internet will affect overall management and patient care. Inflammatory bowel disease (IBD) is a complex illness that affects many aspects of people’s lives. The multifaceted nature of IBD management coupled with the demanding time constraints faced by physicians makes it difficult to exchange all of the information patients need during time-limited medical appointments. This is particularly true at the time of diagnosis when there are many questions to be answered. These timelimited interactions may have resulted in 86% of those with other chronic conditions using the Internet for information and education (4). The Internet has the potential to provide information quickly and be a useful adjunct to medical consultations. However, to be effective, websites should provide up-to-date, accurate, evidence-based information in an understandable format that addresses the questions that are important to patients. A recent survey (5) conducted in an IBD specialty OrIgInal artIcle

Internet répond-il bien aux questions des patients sur les maladies inflammatoires de l'intestin ?HISToRIQue : Internet est une source croissante d'information en santé.oBJeCTIF : Évaluer si les sites Web courants répondent bien aux questions des patients sur les maladies inflammatoires de l'intestin (MII).MÉTHodoloGIe : Les chercheurs ont repéré et évalué 30 sites Web.D'après une étude antérieure sur les besoins d'information des patients, ils ont élaboré une liste de questions fouillées dans les trois secteurs suivants : information médicale (sept points), traitement médical (six points) et autogestion (huit points).Ils ont évalué les sites Web d'après la quantité d'information qu'ils contenaient pour répondre à chaque question au moyen de deux mesures standard de qualité d'information, soit les échelles DISCERN et de garantie d'information de qualité pour les patients.RÉSulTaTS : Quatre sites Web particulièrement solides ont obtenu les résultats les plus élevés (sur une échelle de un à cinq) à l'égard de l'information sur les MII : la Crohn's and Colitis Foundation of America (indice d'information moyen de 4,3), About.com(4,2), HealthCentral (3,8) et WebMD (3,8).Ces sites obtenaient également de bons résultats d'après les échelles DISCERN et de garantie d'information de qualité pour les patients.La plupart des sites Web proposaient de l'information au moins pertinentes sur les symptômes courants, les complications, les traitements et ce que l'on sait (ou que l'on ne sait pas) des causes des MII.Cependant, de nombreux sites ne fournissaient pas d'information pertinente au sujet du pronostic, des effets secondaires possibles des traitements et des risques de cancer.L'information sur l'autogestion était très limitée.CoNCluSIoN : Les sites Web pourraient être renforcés s'ils contenaient plus d'information que les patients jugent importante et s'ils indiquaient plus clairement les sources d'information et la date de leur mise à jour.Dans la plupart des sites Web, il serait bénéfique d'accorder plus d'attention à réduire le niveau de lecture et à améliorer l'organisation de la documentation.clinic indicated that the most common sources of IBD information for patients were a gastroenterologist (59%), the Internet (54%) and a primary care physician (54%).Internet use was higher among younger patients, particularly those younger than 40 years of age and those with higher education.The most commonly used websites were associated with health care institutions or public organizations such as the Crohn's and Colitis Foundation of America (CCFA).Of the 69% who used the Internet to gather information about IBD, 78% indicated that it improved their understanding of the disease, and 77% indicated that it improved their understanding of treatment options.
The goal of the current study was to evaluate the adequacy of information provided by popular websites to answer specific patient questions concerning IBD.Our study complements previous studies (6)(7)(8) evaluating Internet information regarding IBD by assessing a broader range of information areas based on questions that patients consider to be important.We identified these questions in a recent survey of patients diagnosed with IBD within two years (9).Rather than using a checklist that simply identified the presence or absence of information in each area, our aim was to evaluate how well the websites dealt with each topic.A second goal was to evaluate, with established measures of quality, the websites patients were likely to visit.Identifying the strengths and weaknesses of existing websites may contribute to improving the information available to patients and their families in the future.Furthermore, we can direct health care providers to the current best sites (from both an information and a readability perspective) so they can provide this information to their patients.

Website selection
The purpose of the present study was not to identify every possible website with information but, rather, to identify typical websites that would be identified in a search.'Google' was used as the search engine because it is the most widely used search engine in North America and the world by a large margin.Information from specialty industry sources (www.searchengineland.com/comscore-us-most-searches-china-slowest-34217)indicated that Google was used in December 2009 for 67% of searches worldwide, while its closest competitor, 'Yahoo', was used for 7%.Figures for North America were similar.On July 24, 2008, a search was conducted using the key words "inflammatory bowel disease".A refined search was also conducted using the <for patients> link when the list of websites from the first search was retrieved.The first 30 websites that were retrieved through these two searches were considered for inclusion in the study.The first 30 websites of each search were targeted because a recent study (10) of online consumers found that 62% of search engine users click on a search result within the first page of results, and 90% of search engine users click on a result within the first three pages of search resultsnumbers that have both been increasing over the past few years.Exclusion criteria included duplicates, excerpts from books and articles (not websites), non-IBD-related websites and websites containing too little information to be regarded as a comprehensive resource (defined as less than 500 words).Of the 30 websites retrieved in the <for patients> search, 21 were included: five had too little information, one was about irritable bowel syndrome, one was an abstract and two were links to one of the selected websites.Of the 30 websites retrieved in the general search, eight were included, five were duplicates from the <for patients> search, two were links with one of the selected websites, six had descriptions of books, two were for articles and seven had too little information.An additional website -KidsHealth For Kids -was selected because the teen and parent version had been selected in the other searches and was believed to be helpful in evaluating all three.
In applying these selection rules, it is important to note that the websites identified would vary from one time to another depending on Internet activity and the decision rules applied by the search website.Because the search originated in Canada, two Canadian websites were identified that may not have appeared if the search had originated in the US.The search results would also likely differ somewhat if different search terms, such as "Crohn's disease" and "ulcerative colitis", were used.The study was not intended to be an exhaustive evaluation of all websites with information about IBD but, rather, an evaluation of typical information that would be retrieved in a search by a member of the public and, having examined the trends in the websites evaluated, it is most likely that the same issues that were identified would arise across other IBD-related websites also.

Comprehensiveness of IBd information relevant to patients
There are many different types of information about IBD that patients may desire.Based on previous experience and surveys of patient information needs (9), a comprehensive patient question list was developed in the following three areas: medical information concerning IBD (seven items), medical treatment of IBD (six items) and self-management of IBD (eight items).A survey of patients diagnosed with IBD within three to 24 months was administered to patients presenting at six different gastroenterology clinics within the city of Winnipeg, Manitoba.It was found that patients rated information in each of these topic areas as very important (9).The survey sample encompassed a wide range of age and education levels, was balanced in terms of sex and disease type (Crohn's disease versus ulcerative colitis) and consisted of patients with varying degrees of disease activity.The information from each website addressing each topic (excluding external links) was copied into a word processing file for further evaluation.Information on each topic was assessed on a five-point rating scale ranging from 1 (no information), to 3 (adequate information with basic detail), to 5 (complete and comprehensive information with detailed discussion) (11).The ratings for all of the topics were averaged to create a mean information score for each website.The first rater (SP) was a medical student between his second and third year of training (four-year program).He did not have previous experience with IBD and was considered to have a level of knowledge similar to a well-educated patient with a strong science background.To provide an assessment of the completeness of the information from the perspective of a person knowledgeable in the IBD content area, the material on each topic was also evaluated by a gastroenterology fellow (MT) in postgraduate year 6 to assess the degree of agreement on scoring with a person more experienced with the content area.Interobserver agreement was assessed by comparing the ratings of the two evaluators for each question using the intraclass correlation coefficient (12).

Measures of website quality
Overall, website quality was assessed using the DISCERN (13) and Ensuring Quality Information for Patients (EQIP) (14) scales.DISCERN is a 15-question instrument used to assess the quality of consumer health information concerning treatment choices (written materials including material on the Internet).Each question receives a score of 1 to 5, with 1 indicating no success in meeting the question criteria, 2 to 4 indicating partial meeting of the question criteria, and 5 indicating complete fulfillment of the question criteria.The scale has detailed scoring material available on the Internet (www.discern.org.uk).For each website, the scores of the 15 questions were averaged to produce an overall website score out of 5.The DISCERN scale is widely used (including in the three recent studies of Internet information on IBD [6][7][8]) and has good psychometric properties (13,15).EQIP is a 20-question tool specifically designed to assess the presentation quality of health care information.The questions cover language, style and design (including figures), with content answered as 'yes', 'partly' or 'no'.These responses are converted using a formula to produce a per cent score out of 100.

Readability
The reading level of the text from each website was assessed using the Flesch-Kincaid Grade Level score (16), available in the word processing program 'Microsoft Word' (Microsoft Corporation, USA).The Flesch-Kincaid Grade Level score analyzes the relative numbers of syllables, words and sentences in a passage of text to calculate a readability score that indicates the grade level at which a person would be able to understand the text.

ReSulTS
The websites reviewed in the present study are presented in Table 1.The search strategy identified a wide variety of websites including general sources of information for the public (eg, Wikipedia), general health-related websites (eg, eMedicine and WebMD), hospitals and health centres (eg, Cleveland Clinic, Mayo Clinic), a national association (eg, the CCFA) and IBD-specific websites (eg, MayoIBD).Many of the sites focused on adults while others were focused on the needs of children and their parents.The reading grade level of the sites varied considerably with lows in the grade 9 level and highs at *Order is from the 'Google' "for patients" search when not in brackets or from the 'Google' general search when in brackets and with the letter G.Some sites appeared in both searches; † The DISCERN score is a rating from 1 to 5 averaged across 15 questions; ‡ The Ensuring Quality Information for Patients (EQIP) score is based on a scale from 0 to 100; § The KidsHealth for Kids site was evaluated because the parent and teen versions were identified in the general search; ¶ Two websites from the Mayo Clinic resulted from the patient search and were evaluated independently.IBD Inflammatory bowel disease the 13 and 14 years of education level, which could be difficult for significant proportions of the population.Given the different areas of focus of the websites, one would not expect that they would all provide as much detail in responding to the information needs of adults with IBD.On the other hand, it would be reasonable to evaluate the sites according to the DISCERN and EQIP criteria, which are designed to apply to any website.The mean information score for each website was strongly influenced by the number of IBD information topics covered by the website (r=0.92,P<0.01).The relationship between the amount of IBD information provided on the website and quality measures was evaluated by calculating Pearson correlation coefficients.The mean information score correlated with the DISCERN score (r=0.76,P<0.01) and the EQIP score (r=0.54,P<0.01).The DISCERN and EQIP scores were also correlated with one another (r=0.76,P<0.01).The DISCERN instrument is focused on treatment information while the EQIP scale focuses on broader characteristics of the website including its visual appearance and organization.The assessment of interobserver agreement indicated that evaluators had a high level agreement on the mean information score for each website (r=0.94,P<0.01).
A description of the amount of information provided on the websites to answer patients' questions concerning IBD is provided in Tables 2, 3 and 4. The tables present both the proportion of websites providing no information on a particular topic and the mean score (on a scale of 1 to 5).The mean information score was based on the ratings of the first evaluator (SP) across all 30 websites.Considering medical information regarding IBD (Table 2), most websites provided at least adequate information on the common symptoms of IBD (mean rating of 3.4 on the five-point scale), complications (3.3) and what is known (or not known) about the causes of IBD (3.6).On average, less information was provided on long-term prognosis (2.8), risk of developing cancer (2.7), how IBD or medications used to treat it may affect fertility (2.0), or the risk of children developing IBD (2.6).Considering medical treatments for IBD (Table 3), on average, websites provided adequate information regarding medications and surgery, but limited information on possible side effects of medication and on how to manage symptoms of IBD, with only basic details concerning each of these topics provided.Information that patients judge to be important concerning self-management of IBD (Table 4) was covered to a very limited extent by the websites, with modest amounts of information provided about diet and sources of support in coping with IBD.Many of the websites contained no information about self-management.The intraclass correlation coefficient (the rightmost column of Tables 2, 3 and 4) suggested that there was an acceptable level of agreement between the two raters on each topic area.There was one lower value on the topic of 'common symptoms of IBD', with a correlation (r) of 0.50.A comparison of the ratings of the first (SP) and second evaluator (MT) on this topic indicated that their ratings were within one point of one another on 25 of 30 websites and within two points on the remaining five.The low correlation may have been due, in part, to the limited range of ratings on this topic because almost all of the websites achieved a rating of at least 3 (adequate information).Rater MT, a gastroenterology resident, generally rated the sites as having higher amounts of information on the topic of common symptoms of IBD.The information score is based on a five-point rating scale ranging from 1 (no information), to 3 (adequate information with basic detail), to 5 (complete and comprehensive information with detailed discussion).All correlations were statistically significant at P<0.01 To further consider how website information concerning IBD could be strengthened in the future, in addition to providing more information in response to patients' questions, it is useful to consider the criteria for the quality scales.Table 5 lists the criteria in the DISCERN scale for written patient information concerning treatments and the average score across the websites.Considering that the highest score is 5, there was room for improvement in almost every area evaluated.
The EQIP criteria are described in Table 6.There is considerable overlap between these criteria and those in DISCERN.The EQIP scale indicated that most websites use a respectful tone, present the information logically, and explain complex words and jargon, but they may benefit from using shorter sentences.The majority of sites described treatment options and the benefit of treatment; however, only one-half identified the risks and side effects of treatment, and indicated other treatment options.Many websites indicated the date the document was produced, but fewer indicated the source of the information.It is understandable that few websites provided contact information for health services (as is often the case in written information provided in a specific clinic, for example) because it would not be feasible to provide information about local resources on nationally and internationally used websites.Similarly, on websites that may be accessed by a wide range of people, it is not always appropriate to address readers personally because the reader's circumstances are unclear.

EQIP score, mean
The document mentions which subjects will be covered and covers them 0.70 Use of everyday language, explains complex words or jargon 0.75 Use of short sentences (<15 words on average) 0.18 The document personally addresses the reader 0.50 The tone is respectful 0.98 The design and layout are satisfactory (excluding figures or graphs) Each item on the scale was scored on a scale ranging from 0.0 to 1.0, with the values 0 (no), 0.5 (partly) and 1 (yes).The mean was calculated across the 30 websites evaluated.NA Not applicable dISCuSSIoN Four particularly strong websites, which scored highest in terms of IBD information, were the CCFA (mean information score 4.3 out of 5), About.com(4.2), HealthCentral (3.8) and WebMD (3.8).These websites also scored well on the DISCERN and EQIP quality scales.The range of DISCERN scores was similar to that reported in recent studies of IBD websites (7,8).
In considering the reading level determined in our study, there was a broad range in terms of readability scores that were skewed toward a higher grade level.The grade levels in our study were similar to those reported in two recent studies of Internet information on IBD (7,8).It is generally recommended that health information for the public be targeted at a reading level of approximately grade 8 -perhaps even grade 5 -given the low literacy of many people in the community (17).In practice, however, it is difficult to achieve these levels with medical information.Persons with serious literacy problems may struggle to understand written information even when it is at a more basic reading level (18).Developing material at desirable readability levels is complicated by the complex information about IBD that is required, and the use of medical terms that increase the length and difficulty of words.Some of these difficulties may be offset by providing an extensive glossary and illustrations, or videos that can act as easier-to-understand supplements to the written material provided.While it may be difficult to attain grade 8 reading levels, for example, attention to reading level in the continuing development of educational materials is likely to improve quality.The sites with the highest mean IBD information scores tended to have lower reading levels; the one website that focused on children (KidsHealth kids) had a lower reading level.
Tailoring information in a manner that is easily understood by patients ( 19) is a significant challenge in improving the information available to the public via the Internet.Some types of information are more readily available than others.For example, considering medical information about IBD (Table 2), information regarding common symptoms of IBD, complications that may arise, and long-term outcome or prognosis is available in many resources for patients.On the other hand, information on risk of developing cancer may be less clear and should be communicated to patients in ways that are informative without producing undue anxiety.For example, information about cancer might emphasize the minor risk and the importance of surveillance colonoscopy.Definitive information about the cause of IBD is not available; however, written material for patients might discuss some of the areas that are being considered in the research on causes.Given that much about IBD is still unknown, and research in IBD is a particularly active field, it would be helpful for websites to have a plan for regular reviews of their information and to provide the date of the latest review.This would assure patients that the information they are reading is current.Information regarding the effects on fertility of IBD and the medications used in fertility treatment, and the risk that children of parents with IBD have, may be of particular interest to patients of child-bearing age or with children.
Information concerning the medical treatment of IBD (described in Table 3) may also be challenging to communicate to patients in straightforward ways.Furthermore, it is likely that not all experts would agree on the best recommendations in each of these areas (such as whether medication should be adjusted without consultation with the physician and when to contact the physician).While information on medication side effects may be widely available, patients are often concerned that this information is communicated in ways that are difficult to interpret, such as listing side effects without indicating the proportion of patients who are likely to experience them (19).Similarly, while patients rated many topics concerning selfmanagement of IBD to be very important (Table 4), adequate answers may not be available for all of these questions, and experts may disagree about the best advice to provide.
The description of treatments on many websites would be strengthened by providing additional information concerning the benefits and risks of the common treatments, how they work, treatment choices, how treatments influence quality of life and areas in which there is some uncertainty concerning treatment.It would also be helpful to provide information about the sources of information used in development and when this information was produced.
A limitation of the present study is that the websites identified for evaluation were selected on a particular day and using the specific search term "inflammatory bowel disease".A search performed one year later, or using alternative search terms such as "Crohn's disease" and "ulcerative colitis", would identify different websites.However, we believe that the quality issues identified in the present study apply to a wide range of possible websites, as suggested by a comparison between our study and other research on Internet information.In comparing our findings with those of recent studies of Internet information on IBD (6-8), there were several similarities.A group from The Netherlands (8) evaluated English language websites using a 35-point checklist based on European evidence-based consensus statements concerning the diagnosis and management of IBD.While these websites, many of which were European based, frequently covered definition of disease, signs/ symptoms and causes, more than 70% were missing information deemed to be important on a number of both medical and surgical management issues.A team from Dalhousie University (Halifax, Nova Scotia) and the University of Alberta (Edmonton, Alberta) (7) developed a detailed measure of the information provided on Crohn's disease and ulcerative colitis based on a review of the literature pertaining to patient concerns and the opinions of gastroenterologists.The recent study by this group focused on a detailed analysis of information provided about medication treatments and, as with The Netherlands study, among the websites reviewed, the majority did not provide sufficient information on many aspects of medical management deemed to be important (7).The CCFA and WebMD websites were identified by all of the studies to be in the top six for information content.In spite of the different websites identified, the studies identified similar strengths and weaknesses in the patient-oriented information available on the Internet.Information concerning symptoms, medical treatments and surgical treatments was more often covered adequately.Information concerning prognosis, risk of cancer, medication side effects and management of symptoms was less adequately covered.Our study provided additional information on issues of self-care and quality of life -issues that are particularly important to patients.Many of the websites provided very little information in these areas.All of the studies found large variations in the quality and comprehensiveness of websites.The best quality websites were often not among the first 10 identified by the Internet search engine.Taken together, our study and other recent studies identify a number of areas in which the quality of patient-oriented websites could be improved.
In developing information and decision aids for patients, many approaches have been developed that are likely to be useful.The International Patient Decision Aid Standards Collaboration (20) described criteria that are important in developing patient decision aids, and can also be applied to developing and evaluating patient educational material.Key processes in this area include determining what information patients need and want, developing educational material based on available evidence, arranging for peer review of draft material by patients and professional experts not involved in the development of the educational materials, and field testing of materials by users and practitioners.
When dealing with a health question or concern, the majority of people continue to see a health professional, usually their physician, as a key source of information (1).The use of the Internet as a source of information is steadily increasing, particularly with more frequent access using wireless devices.High-quality, comprehensive, unbiased information on the Internet has the potential to complement the information provided in consultation with health professionals and to enable people to be more active in caring for their health.aCkNoWledGeMeNT: The authors thank Kristin Reynolds for her assistance with data analysis.
Figures or graphs are clear and relevant 0.66 Information is presented in a logical order 0.87 The document has a designated space for the reader's notes NA Specific contact details for health services 0.22 Date of issue or revision provided 0.72 Names the persons or organization that produced the document

dISCloSuRe:
Steven Promislow was supported by a Mindel and Tom Olenick Scholarship.Charles Bernstein is supported in part by a Crohn's and Colitis Foundation of Canada Research Scientist Award and the Bingham Chair in Gastroenterology.In the past year, Dr Bernstein has served on the advisory board of Abbott Canada and has received an unrestricted educational grant from Axcan Pharma.