Improvement in Accuracy and Concordance of American Society of Anesthesiologist's Physical Status (ASA-PS) Scoring Assignment over a 11 Year Time Period Using Patient BMI as a Comorbidity Finding

Background Anesthesia providers categorize patients utilizing the American Society of Anesthesiologists Physical Status (ASA-PS) classification originally created by the ASA in 1941. There is published variability and discordance among providers when assigning patient ASA scores in part due to the subjectivity of scoring utilizing patient medical conditions, but variability is also found using objective findings like BMI. To date, there are few studies evaluating the accuracy of anesthesia providers' ASA assignment based on objective body mass index (BMI) alone. The aim of this retrospective chart review is to determine improvement in accuracy of anesthesia providers to correctly assign patient ASA scores, based on BMI criteria added to the ASA-PS in October of 2014, utilizing a multifaceted strategy including creation of an active finance committee in the fall of 2015, multiple e-mail communications about the updated definitions and recommendations for ASA-PS scoring in the fall of 2015 and spring of 2016, a department grand rounds presentation in February 2016, placement of laminated copies of the ASA definitions and recommendations in the anesthesia chartrooms, and the development of a tool embedded into our EMR providing a recommendation of ASA-PS based on patient comorbidity findings. Methods After attaining IRB approval, all eligible patients over the age of 18 who had surgical procedures under general anesthesia at Mayo Clinic in Rochester, MN, between January 1, 2010, and December 31, 2020, were retrospectively analyzed. A segmented logistic regression model was used to estimate the trends (per-year change in odds) of ASA under classification according to severity of obesity during 3 epochs: preimplementation (2010–2014), implementation (2015), and postimplementation (2016–2020). Results A total of 16,467 patients of the 200,423 (8.2%) patients with obesity (class 1, 2, and 3) were underscored based on BMI alone. Accuracy of ASA-PS classification, as it pertains to BMI alone, was found to show meaningful improvement year-to-year following the updated ASA-PS guidelines with examples released in October of 2014 (P < 0.001). Most of the improvement occurred in 2015–2017 with relatively little between-year variability in the rate of underscoring from 2017–2020. Conclusion Despite updated ASA-PS published guidelines, providers may still be unaware of the updated guidelines and inclusion of examples used within the ASA-PS classification system. Accuracy of scoring did improve annually following the release of the updated guidelines with examples as well as department-wide educational activities on the topic. Additional education and awareness should be offered to those responsible for preanesthesia evaluation and assignment of ASA-PS in patients to improve accuracy as it pertains to BMI.

Studies looking at inter-rater reliability and concordance of ASA-PS have consistently shown wide variation in scoring between anesthesia and nonanesthesia providers alike.For example, Owens et al. found signifcant discordance in scoring using the ASA-PS when 304 randomly picked anesthesiologists were sent surveys asking them to classify patients using the ASA-PS defnitions, as less than 60% of patients were scored identically [6].One of the most perplexing causes of potential scoring discordance is body habitus.Obesity has long been understood as a complicating factor for surgical outcomes and the management of anesthesia [3,16,18].However, there is a paucity of research to date to evaluate the accuracy of ASA-PS in scoring patients based on obesity.
On October 15, 2014, and again on December 13, 2020, the ASA Committee on Economics released updated ASA-PS guidelines that included examples of various physical states and comorbidities which now include adult, pediatric, and obstetric examples [5].Te new guidelines listed several examples of conditions to help reduce subjectivity and provider-provider discordance in scoring, including examples of obesity.Class 1 and 2 obesity are grouped together in an ASA-PS category using body mass index (BMI) defned as BMI 30.0-39.9 kg/m 2 and carry a minimum ASA-PS of II.Class 3 obesity is defned as BMI >40.0 kg/m 2 and carries a minimum ASA-PS of III.Utilizing these examples, Fielding-Sing et al. found that 4.8% of patients with class 1 or 2 obesity and 27.6% of patients with class 3 obesity were underclassifed by ASA-PS [21].Teir study suggested that the published examples may have had a limited impact on improving accuracy and concordance.
We hypothesize that anesthesia and nonanesthesia providers at our institution underscore patients based solely on BMI.Additionally, we hypothesize that, unlike the fndings from Fielding-Sing et al., the publication of examples by the ASA, as well as educational eforts, will have improved the accuracy and concordance of ASA-PS assignment over the eleven-year time period of this study.In 2015, our department started an initiative to improve the accuracy of ASA-PS assignment using a multipronged approach.We developed a fnance committee with clinical, administrative, and revenue cycle representation that disseminated several emails in the fall of 2015 and spring of 2016 to department leaders highlighting the historical discordance in ASA-PS and its potential fnancial impact.We also had a department grand round dedicated to this topic in February 2016, placed laminated copies of the ASA defnitions to assist with ASA-PS scoring in the spring of 2016, and developed a tool embedded within our EMR that assists the clinician with reviewing the patient's history and provides a recommended ASA-PS score based on identifed comorbidities (Figures 1 and 2).

Methods
Tis retrospective observational study was deemed exempt from need to obtain informed consent by the Mayo Clinic Institutional Review Board.All patients over the age of 18 who consented to use of their clinical data and had surgical procedures under general anesthesia at Mayo Clinic in Rochester, MN, between January 1, 2010, and December 31, 2020, were retrospectively analyzed.Variables collected included age, height (cm), weight (kg), BMI, ASA-PS, and type of surgery performed.

Statistical Analysis.
Patient characteristics are summarized using mean ± SD for continuous variables and frequency counts and percentages for categorical variables.A dichotomous variable was created to indicate patients whose ASA status was underscored based on obesity criteria.Patients with BMI ≥30.0 kg/m 2 were considered underscored if their assigned ASA status was less than 2, and patients with BMI ≥40.0 kg/m 2 were considered underscored if their assigned ASA status was less than 3. Te percentage of patients underscored based on obesity criteria was calculated using diferent denominators (i.e., diferent patient populations) including all patients, all obese patients, and BMI subgroups of 30.0 to 39.9 kg/m 2 and ≥40 kg/m 2 .Te percentage of patients who were underscored is summarized overall and according to calendar year.We ft a segmented logistic regression model to estimate the trends (per-year change in odds) of ASA under classifcation according to the severity of obesity during 3 epochs: preimplementation (2010-2014), implementation (2015), and postimplementation (2016-2020) [25].

Results
Over our study period, 495,7779 patients were identifed as having a procedure under general anesthesia at our institution with all available data present.Table 1 shows patient demographics and breakdown of patients based on BMI.Demographic data show equal distribution of males (50%) to females (50%) in the general population studied.Table 1 shows further breakdown of gender and its correlation to each level of obesity.Te largest gender discrepancy was in the BMI >40.0 category where the population was 39% male and 61% female.
Of the patients included in the study, 160,844 (32.4%) met the criteria to be classifed as a minimum ASA 2 based on BMI criteria (BMI 30.0-39.9 kg/m 2 ), and 39,579 (8.0%) met the criteria to be classifed as a minimum ASA 3 based on BMI criteria (BMI >40.0 kg/m 2 ).Of the 160,844 patients 2 Anesthesiology Research and Practice  with class 1 or 2 obesity (BMI 30.0-39.9 kg/m 2 ), minimum ASA-PS 2, a total of 5,695 (3.5%) patients were underscored.
When breaking down data year-to-year following the release of the updated ASA-PS in 2014, there was a statistically signifcant improvement in underscoring of patients based on BMI criteria alone (Table 2) from 2015 to 2020 in all obesity categories (P ≤ 0.001).Despite the improvement year-to-year in the population studied, patients undergoing general anesthesia during the study period of 2020 continued to be underscored.Tose patients in 2020 who met criteria for class 3 obesity continued to be underscored at a higher rate (15.8%) than those with class 1 or 2 obesity (2.6%).
Tere was a statistically signifcant per-year increase in the odds of underscoring of ASA-PS based on BMI alone from 2010 to 2015 (P < 0.001); however, between-year variability was low (maximum rate minus minimum rate: 2.6%, 1.0%, and 1.5% among class 3, class 1-2, and all obese patients, respectively).
Improvement in scoring can be found after 2015 (Figure 3).

Discussion
Tis study shows that the ASA-PS continues to be underscored based on obesity, particularly for class 3 obesity (BMI >40.0 kg/m 2 ).Additionally, there does seem to be a trend in improving the accuracy of ASA-PS scoring over time.Te 2014 and 2020 ASA-PS updated guidelines specifcally state minimum scores for patients with class 1 and 2 obesity (ASA II) and class 3 obesity (ASA III) [5].Tese represent a foor or minimum ASA-PS, but the determined score may be higher depending on other patient characteristics or comorbidities.Tus, the problem of underscoring maybe even more substantial than this study indicates, as there could be patients with class 3 obesity that were scored as ASA-PS 3 based on other comorbidities but would have been scored lower if those comorbidities were not present.
Our study adds to a growing body of literature on the subject, as previous studies looking at the concordance of scoring using the provided examples have yielded mixed results.Several articles have suggested inconsistencies and discordance in assignments among both anesthesiologists and nonanesthesia providers [17,19].Hurwitz et al. [15] created a web-based questionnaire using 10 hypothetical cases in which both anesthesia-trained and nonanesthesiatrained clinicians were asked to assign an ASA-PS score in case scenarios using the ASA-PS classifcation system, and then rescoring a second time using ASA-PS approved examples and defnitions.Te authors demonstrated that providing publicly available ASA-approved examples signifcantly improved concordance of the assignment of the ASA physical status by both anesthesia-trained and nonanesthesia-trained providers.Te number of correctly assigned cases increased from 5/10 to 7/10 cases after examples were provided for anesthesia-trained providers and 8/10 for nonanesthesia-trained providers.Not evaluated was a longitudinal evaluation of whether the examples improved concordance and accuracy.
Variability studies using kappa scores have further quantifed the variation between providers with both theoretical [7,9,11,16,20] and actual patients [13,14,17,22].Much of the discordance is attributed to the subjective identifcation of patient comorbidities and the severity of each condition.However, there are several objective variables listed on the current ASA-PS Classifcation System Current Defnitions and ASA-Approved Examples website.A few of the provided examples include objective data such as BMI thresholds, cardiovascular or cerebrovascular events within a defned time period, age of infants, or specifc comorbidities such as uterine rupture, sepsis, dialysis, or COPD.It is important to note that the list of examples in the  20) 58 ( 18) 59 ( 15) 55 ( 14) Gender † guidelines released in 2014 and 2020 is not comprehensive and not intended to supersede critical evaluation and designation by trained providers.
We are encouraged by the signifcant improvement in concordance and accuracy of ASA-PS scoring over time.In 2015, our institution undertook an educational initiative on this topic which included a grand round presentation, multiple conference presentations, a new Department fnance committee, and work on an IT solution to provide a recommended PS and provided copies of the ASA-PS examples throughout our anesthetizing locations.Signifcant improvement in PS scoring based on BMI was seen from 2015-2017, but, without continued active educational activities, there has been no further improvement over the last several years.Tis represents an opportunity to reinstitute educational activity to encourage further improvements in accuracy and convergence of ASA scoring on the day of surgery.
Te use of an automated software tool or ASA-PS scoring algorithm within an electronic record may be useful in lessening the number of patients who are underscored, based on fxed criteria, such as BMI.Such automated methods and systematic approaches to the use of the ASA-PS have been reported and are in development, including one within our institution.Further educational opportunities or IT solutions may be benefcial to help reduce the percentage of patients underscored using the ASA-PS based on BMI as well as other comorbidities.

Conclusion
Despite the updated ASA-PS released in 2014 and again in 2020 that included defnitions and examples specifc to ASA-PS and BMI [4,5], our study shows there continues to be an underscoring of patients based on objective BMI data alone.Rates of underscoring in general and correlation to increased rate of underscoring in patients with BMI >40 kg/ m 2 are consistent with a large national study looking at the accuracy of providers scoring utilizing BMI [21].Te use of the ASA-PS to classify patients preoperatively has many potential implications and is used by anesthesia-trained and nonanesthesia-trained providers in a wide variety of settings in traditional ORs, procedural spaces, and additional environments utilizing both anesthesia and nonanesthesia providers.Additional education, tools, and awareness on the use of the updated ASA guidelines and examples, especially on more objective criteria such as those listed for BMI, may be warranted to help improve overall scoring accuracy.Future studies focusing on IT solutions and utilization of software to identify clinical data, patient characteristics, and ICD 10 codes may also allow for less subjectivity and improve accuracy and concordance in the assignment of scores.Anesthesiology Research and Practice 5

Figure 1 :
Figure 1: Image from our EMR preanesthesia note creator on a sample patient showing a hyperlink to the ASA scoring tool clinicians may utilize to assist with ASA-PS scoring.

Figure 2 :
Figure 2: Comorbidities identifed by the scoring tool from a sample patient and a recommended ASA-PS score generated by the tool.Te score may be used by the provider with the ASA-PS assignment if they are in agreement with the recommendation.

Table 1 :
Patient characteristics overall and according to body mass index category * .

Table 2 :
Number and percentage of patients with ASA status underscored based on obesity criteria in patients undergoing general anesthesia * .Te number and percentage (# (%)) of patients underscored are presented overall, according to year of surgery.Data are presented for all patients, all obese patients, patients with a BMI of 30.0 to 39.9, and patients with a BMI of 40.0 or greater. *