Analysis of Negative Reviews of Orthopedic Oncology Surgeons: An Investigation of Reviews from Healthgrades, Vitals, and Google

Background Physician review websites (PRWs) are increasing in usage and popularity. Our purpose is to characterize one-star reviews of orthopedic oncology surgeons to understand factors in healthcare that contribute to patient satisfaction. Methods Orthopedic oncology surgeons were randomly selected from the Musculoskeletal Tumor Society. A search for one-star reviews was performed on Google Reviews, Healthgrades, and Vitals.com. Reviews were classified as clinical or nonclinical. Statistical analyses were performed regarding the frequency of reviews and complaints for each category. Results Of the 7,733 reviews discovered, 908 (11.7%) were identified as one-star reviews. Of 907 usable complaints, 362 (40.8%) were clinical and 545 (59.2%) were nonclinical. The most common nonclinical complaints included bedside manner (65%) and limited time with providers (19%). The most common clinical complaints included complications (26%) and disagreements with the treatment plan (26%). There were 120 surgical and 221 nonsurgical reviews. Surgical patients had a higher rate of clinical complaints. Nonsurgical patients had a higher rate of total complaints. Conclusion To the best of our knowledge, this is the first study examining PRWs regarding orthopedic oncology surgeons. Most one-star reviews were due to nonclinical complaints from nonsurgical patients. The most common factors are bedside manner, limited time with provider, phone communication issues, and rude/unprofessional conduct.


Introduction
As technology continues to integrate into healthcare, there is a progressive increase in virtual and online medical resources [1]. One particular aspect of this movement toward online resources includes physician review websites (PRWs), which have grown increasingly popular in recent years [2]. Specifcally, it was reported in 2017 that 39% of patients utilize these resources [3]. Tese platforms also hold value for patients, as it has been reported that 60% of Americans believe online reviews of physicians are either somewhat or very important in helping them seek out healthcare services [2]. Further, over 50% of patients using online reviews have eliminated specifc healthcare providers from their list of options based on the content of online reviews [4].
Te emergence of online review sites has been encouraged by the policy in addition to patient preference. Te changes mandated by the Afordable Care Act of 2010 created a shift toward patient-centered and value-based outcomes in medicine. In collaboration with patient empowerment, an update by Consensus Core Set of Orthopedic Measures (CCSOMs) was released to simplify value-based measurements [5]. One principle outlined in the CCSOM was "patient experience," which prioritized patients' selfreported outcomes over more traditional objective measures of health outcomes. To incentivize providers to meet specifc patient quality benchmarks, hospitals have implemented reimbursement models that heavily weigh these subjective patient experiences, launching a new era of value-based care physician compensation. Tis is commonly measured and reported with the publicly available Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Surveys, which use a standardized survey instrument and data collection methodology that is used to measure patients' perspectives of hospital care. With the above in mind, it has become increasingly clear that patients' opinions are becoming more valuable.
Despite their popularity, the utility of these patient-created reviews is still highly controversial [6][7][8][9]. Understandably, as these reviews afect compensation, patient-physician relationships, and potential patients, physicians have voiced concerns about the validity of these reviews [10]. Despite the controversy, it is vital that physicians have a thorough understanding of their online reviews and what may lead to a poor online reputation. While there are prior studies analyzing orthopedic surgeons' online ratings, few have examined the reasons for negative reviews. In Bernstein's review, most of the studies included that evaluated PRWs of various orthopedic specialties analyzed presence of positive reviews and average physician scores [11]. In addition, there have been no studies investigating reviews or ratings of orthopedic oncologists [11,12]. Te aim of this study is to characterize one-star reviews of orthopedic oncology surgeons to further understand factors that afect the patient experience in this unique patient group. Tis information will allow orthopedic oncology surgeons to further understand what factors are most considered by patients and identify targets to improve their online reputation.

Methods
A random number generator was utilized to select 200 of the 265 surgeons listed on the Musculoskeletal Tumor Society (MSTS) "Member Search" tool on the MSTS website [13]. We decided on 200 surgeons as we believed this would capture an appropriate sample size of reviews. A search was then performed for all reviews listed under the MSTS surgeon's name on the following websites: Google Reviews, Healthgrades, and Vitals.com. Tese three websites were chosen as they have been listed by the reputation industry as the most important for physicians to consider [14]. Only one-star reviews were collected; all other reviews (out of a possible 5 stars) were excluded from this study. In addition, for each of the three websites, a surgeon's average rating (1)(2)(3)(4)(5), number of reviews, and number of 1-star reviews were recorded.
One-star reviews were then classifed by two separate authors (LQ and VE) as either clinical or nonclinical. Nonclinical reviews included comments referring to physician professionalism, midlevel professionalism, front-desk professionalism, time with provider, wait time, facilities, cost, billing issues, phone communication, or scheduling issues. Clinical reviews included comments with reference to complications, readmission, reoperation, pain, misdiagnosis, disagreement with clinical plan, unclear clinical plan, or delay in care.
Reviews that directly referenced a surgical aspect of care were additionally classifed as operative. Reviews that could not be classifed according to the above criteria were excluded (e.g., "Tis doctor is a disgrace to his profession!"). In addition, any one-star review that was overwhelmingly positive was also excluded (e.g., "Doctor is an amazing physician. I have seen him over the past 10 years and he has taken excellent care of me"). Reviews were classifed and categorized by two separate authors with a third author (ER) to resolve conficting classifcations or categorization between the two reviewers (95.46% agreement and Cohen's kappa statistic of 0.667).
Categorical variables were analyzed using the chisquare test with an alpha set to 0.05. Te rate ratio (the ratio of the rate for nonsurgical reviews divided by surgical reviews) was determined for each category. Inter-rater reliability was calculated between the reviewers utilizing Cohen's kappa statistic. A statistical analysis was performed using a commercially available software package (Microsoft Excel, Redmond, WA).

Results
A total of 7,733 reviews were discovered. Of these, 908 (11.7%) were identifed as one-star reviews. A total of 524 reviews (57.7% of one-star reviews) were excluded as they did not contain any comments to be classifed. Of the 384 one-star reviews with comments, 19 were excluded as they were duplicate reviews, 20 were unclassifable, 2 were positive reviews, and 2 were for the wrong specialty. Tis left 341 one-star reviews for classifcation. In total, these 341 one-star reviews comprised a total of 907 complaints. Each review had an average of 2.57 complaints ( Figure 1).

Rating.
Of the 200 oncology orthopedic surgeons included, 183 (91.5%) were rated at least once. Google.com was the least utilized website for reviews, with 138 (69%) surgeons having at least one review. Te average online rating was 4.44 out of 5 for all surgeons (Table 1). Te average number of reviews per surgeon was 38.7, and the average number of one-star reviews per physician was 4.54.

Clinical vs. Nonclinical.
Of the 907 total complaints, 362 (40.8%) were clinically related, while 545 (59.2%) were nonclinical in nature. Of the 341 one-star reviews, 120 (35.2%) were from surgically treated patients, and 221 (64.8%) were from nonsurgical patients. Surgical patients had a signifcantly higher rate of clinical complaints than nonsurgical patients (1.89 vs. 0.61 clinical complaints per review, p < 0.001). Conversely, nonsurgical patients had a signifcantly higher rate of nonclinical complaints than surgical patients (1.42 vs. 1.70 nonclinical complaints per review, p < 0.001). Nonsurgical patients had a signifcantly higher rate of total complaints per one-star review than surgical patients (1.76 vs. 0.78 nonclinical complaints per review, p � 0.034) ( Table 2).
Te diference in the number of complaints for surgical vs. nonsurgical patients was statistically signifcant (p < 0.05) for multiple complaints (Table 4).

Discussion
Tough many orthopedic surgeons believe that one-star reviews are common on PRWs [10], our study showed that one-star reviews make up only 11.7% of the 7,733 reviews submitted for orthopedic oncologists on Google Reviews, Healthgrades, and Vitals.com. Nonclinical complaints were approximately 1.5 times more common than clinical complaints in one-star reviews of orthopedic oncology surgeons. Te most frequent nonclinical complaints were related to physicians' bedside manner, followed by limited time with the provider and phone communication issues. Te most common clinical complaints included complications, disagreement with the plan, and uncontrolled pain. Tese fndings are similar to those of other studies in which nonclinical complaints were more common than clinical complaints [15][16][17].
While physicians rely heavily on clinical outcomes to guide clinical practice and goal-directed improvements, research suggests that patients rely on nonclinical factors in determining the overall quality of their care. A survey investigating patients' perspectives of physicians showed that patients' views of physicians are more heavily infuenced by the doctor-patient relationship than health outcomes [18]. As refected in our study, the single most frequent complaint was in regard to physician bedside manner, which was present in 65% of the 341 reviews studied. Tese fndings have been echoed by numerous other studies examining Single star reviews for a certifed orthopedic trauma surgeon N = 908 reviews Reviews included in analysis N = 341 classifiable reviews with comments Reviews above one star excluded N = 6,825 single star reviews Reviews with no comments excluded N = 524 reviews without comments Single star reviews for a certifed orthopedic surgeon with a comment N = 384 reviews with comments Reviews excluded that could not be classifed, or duplicates N = 43 unclassifiable or duplicate reviews Google Reviews, Healthgrades, and Vitals.com were searched for reviews listed under 200 randomly chosen MSTS accredited surgeons    [19,20]. Nonclinical complaints regarding phone communication issues have also been previously reported as a common source of patient complaints about physicians [21][22][23]. Similar to bedside manner, quality of communication has a larger impact on patient satisfaction than quality of care as well [8]. Tese studies emphasize the results from our study, suggesting that physicians can dramatically improve patient satisfaction through improvements in bedside manner, open communication, and access through telephone encounters. In our study, complaints of not enough time spent with the provider were second to bedside manner. A study that reviewed 2,185 reviews of orthopedic surgeons from four PRW sites found that time spent with the patient was one of the fve factors that was statistically signifcant in predicting online physician ratings [24]. Another study analyzed 712 online reviews from two rating websites for primary care doctors and found that the majority of Internet reviews of the primary care physicians were positive, but the care encounter extended beyond the physician-patient experience and was dependent on staf, access, and convenience, including shorter wait times [22]. In combination with our study, these results suggest that orthopedic oncologists could better connect with their patients and further reduce one-star reviews and complaints by increasing time spent with patients and thoroughly answering questions during patient encounters. While orthopedic oncologists have limited time due to complicated patients, busy clinic schedules, and various other commitments, physicians who ensure patients feel addressed may beneft from higher online scores and more positive reviews.
Another aspect of the care provided by surgeons that must be considered when attempting to understand negative reviews is the role of the staf who represent the practice. A recent study by Manning et al. reviewed patients' perspectives of midlevel providers in orthopedic sports medicine and found that 62.9% of patients thought a physician's midlevel provider is an important consideration when choosing a physician [25]. Another study analyzed 11,527 reviews by patients of total joint arthroplasty surgeons and revealed signifcant correlation between physician ratings and staf friendliness, punctuality, and knowledge/expertise [26]. Tese results suggest that a physician's online reputation is heavily infuenced by all members of the physician's team, such as nurses, ofce staf, medical assistants, and midlevel providers. In this study, we also found that 4% of complaints were related to midlevel providers' bedside manner and 15% were due to rude/unprofessional staf. Multiple reviews included complaints about ancillary staf such as surgery schedulers that negatively impacted the patients' experiences with the orthopedic clinic. For example, one of the reviews read, "I met with (the doctor) who was pleasant, professional, and seemed quite competent. My problem was with the ofce staf, (the scheduler) in particular. I was strung along for a month waiting for my procedure to be scheduled." Although surgeons should focus on maintaining strong physician-patient relationships, it is important to acknowledge the contributions of staf in representing the clinic and physician themselves. Tese examples accentuate why physicians should encourage pleasant staf interactions and appropriate ofce response time to phone calls to improve patient trust.
Although nonclinical complaints were more common, clinical complaints still comprised 40.8% of all reviews. Te three most common clinical complaints were complications, disagreement with a decision/plan, and uncontrolled pain. Complications, such as infection, are not uncommon in orthopedic surgery, particularly in oncology, given the complexity in patients and procedures. As reported, prevalence of infection after orthopedic surgery ranges from 0.7 to 22.7% [27,28]. However, a study of 1,304 patients who underwent orthopedic oncology procedures for primary bone tumors had an infection rate of 10.1% [29]. Surgical site infection can lead to readmissions, reoperations, and increased costs. Unfortunately, complications are an inherent risk of all surgical procedures. However, not all complications should necessarily result in negative online reviews. One way to improve patient satisfaction despite complications is to clearly communicate risks and benefts while providing expectation management by the physician prior to surgery. In order to improve communication and increase successful medical encounters in orthopedics, the American Academy of Orthopedic Surgeons (AAOSs) created information statements to encourage providers to sit down during patient encounters, show empathy and respect, and involve patients in decisions concerning their medical care [30] as is supported in other resources as well [31,32]. Tese   Our study found that complaints varied greatly depending on whether patients underwent surgical intervention or not. In our study, patients who had a surgical aspect to their care accounted for 35% of onestar reviews, while nonsurgical patients accounted for 65% of one-star reviews. Furthermore, the only nonclinical complaints more common among surgical patients than nonsurgical patients were the lack of time spent with the provider and that wait times were too long. Providers should keep in mind that there is a higher risk of patients leaving one-star reviews if they do not undergo surgical care [33]. Te exact reasons for this association are unclear, but it may represent the follow-up with surgical patients that leads to a longer and potentially stronger relationship. Future studies are necessary to fully understand the relationship between surgical care and patient satisfaction.
Te accuracy of information provided in online review and rating websites for representing quality of care remains controversial. Tere have been prior attempts to better understand the dynamics between online reviews and readmission rates, infections, mortality, and morbidity. However, the results among studies have been inconsistent or have shown poorly supported correlations between online ratings and objective health care outcome measures [19,34,35].
In addition, credibility is questioned as PRW reviews can be submitted anonymously in a way that cannot be verifed [10,36]. As these PRW sites allow patients to autonomously comment in this public space, any person can create an account and protect their identity.
A survey by C. S. Mott Children's Hospital revealed that two thirds of parents thought some online reviews were "fake" and were not reliable to infuence clinical decisions [37]. Tere are also concerns that some reviews are intentional defamation and unjustifed criticism of the physician. In our study, only 1.2% of reviews were classifed as defamation without constructive criticism for the surgeon or physician's ofce. Surgeons should consider that many of the one-star reviews contain constructive criticism helpful to their job performance, while the minority are baseless complaints.

. Limitations
Tis study has several limitations, the frst being that there are many PRWs available. While we have three of the more commonly used PRWs (Google Reviews, Healthgrades, and Vitals.com), our data represent only the PRWs we sampled from. Terefore, negative reviews could present with diferent compositions and frequencies among diferent review websites. In addition, it has been noted that only extremely satisfed or extremely dissatisfed patients are likely to write reviews. In efect, patients are self-selected to leave negative reviews on physician review websites. In contrast, it has been reported that fewer negative reviews are submitted for providers who initiate reviews with their patients by providing their own surveys [36]. Given this information, it is possible that patients who write extremely negative reviews might not portray an accurate representation of surgeons' practices. In addition, some patients might be extremely dissatisfed but not submit a review to Google Reviews, Healthgrades, or Vitals.com. Also, it is possible that the reviews evaluated in this study were incorrectly categorized. However, this probability was discarded as two authors independently categorized the variables and a third author resolved any disparities.
Another limitation to discuss is the fact that we chose to only include one-star reviews from the sampled websites instead of including two-star reviews in addition to one-star reviews. While this would increase the number of reviews, we chose to focus only on one-star reviews in order to serve as a control and standardization across websites. Tough two-star or three-star reviews may be considered negative, that is not universal across websites. For example, Vitals.com provides a three-star scoring system where a three-star review is considered "good" and thus it could be inferred that a two-star review from Vitals.com is not of equal negativity to a two-star review from a website that uses a wider range scoring system. Narrowing the study to include only onestar reviews was chosen to serve as a control to focus on only the most negative reviews to emphasize the purpose of our study.

Conclusion
Our study of extremely negative reviews of orthopedic oncology surgeons gathered from Google Reviews, Healthgrades, and Vitals.com found that most complaints were nonclinical in nature. Te most common clinical complaints were complications, disagreement with the plan, and uncontrolled pain. Approximately one-third of one-star reviews were by patients who had undergone surgery. In our study, the most common complaints were about physician bedside manner, followed by not enough time spent with the provider, phone and communication issues, waiting time, and scheduling issues. Patients who underwent surgery were more likely to have clinical complaints, the most common being complications, disagreement with treatment plan, and uncontrolled pain. An understanding of factors contributing to negative online reviews can provide the feedback necessary for orthopedic oncology surgeons to improve patient satisfaction and increase trust between physicians and patients. Tese results reinforce the notion that orthopedic oncology surgeons can improve patient satisfaction with improved communication and an overall positive clinic experience.

Data Availability
Te data that support the fndings of this study are openly available in Google Reviews, Healthgrades, and Vitals.com. Sarcoma 7