Development of Evidence-Based COVID-19 Management Guidelines for Local Context: The Methodological Challenges

Background The coronavirus disease 2019 (COVID-19) pandemic has presented as a therapeutic challenge for clinicians worldwide due to its rapid spread along with evolving evidence and understanding of the disease. Internationally, recommendations to guide the management of COVID-19 have been created and updated continuously by the WHO and CDC, which have been locally adapted by different countries. Similarly, Pakistan's National Command Operation Center (NCOC), in its national COVID-19 management strategy, generated guidelines for national implementation. Keeping the guidelines updated has proved challenging globally and locally. Here, we present a summary of the process to assess the evidence, including a time-restricted systematic review based on NCOC Clinical Management Guidelines for COVID-19 Infections v4 published on 11th December 2020 version, correlating it with current recommendations and with input one of the guidelines authors, particularly noting the methodological challenges. Methods We conducted a systematic review synthesizing global research on treatment options for COVID-19 hospitalized patients, limiting it to pharmacological interventions for hospitalized COVID-19 patients included in Pakistan's NCOC's national guidelines v4 published on 11th December 2020. Each treatment recommendation's strength and quality of evidence was assessed based on the grading of recommendations assessment, development, and evaluation (GRADE) methodology. These were then compared to the most current living WHO COVID-19 pharmacological treatment guidelines v7.1. One of the authors of the NCOC guidelines reviewed and commented on the findings as well. Results We note that the data from our systematic review strongly supports corticosteroids use in treating severe and critically ill COVID-19 hospitalized patients correlating with WHO v7.1 guidelines 24 September 2021. However, evidence from our review and WHO v7.1 for the use of tocilizumab had some conflicting evidence, with data from our review until December 2020 supporting only a weak recommendation for its use, compared to the strong recommendation by the WHO for the use of tocilizumab in patients with severe or critical COVID-19 infection. Regarding the use of antibiotics and ivermectin use in treating COVID-19 hospitalized patients, data from our review and WHO v 7.1 recommend against their use. Conclusion Research data about the efficacy and safety of pharmacological interventions to treat hospitalized patients with COVID-19 are rapidly evolving, and based on it, the evidence for or against recommendations changes accordingly. Our study illustrates the challenges of keeping up with the evidence; the recommendations were based on studies up till December 2021, and we have compared our recommendations with the WHO v7.1, which showed some significant changes in the use of pharmacological treatment options.


Introduction
Pakistan, like other countries worldwide, has seen many cases of coronavirus disease 2019 (COVID-19) since the pandemic began [1]. e national government-led response included the creation of a central National Command Operation Center (NCOC), setting up designated hospitals, isolation testing facilities, and following dedicated treatment guidelines based on WHO recommendations. National guidelines were created, 2nd April 2020 (v1), with most recent version (v4) currently in use, "Clinical Management Guidelines for COVID-19 Infections" published by the Government of Pakistan on 11 th December 2020 [2]. ese have been formulated by national experts incorporating international guidelines and adapting them to local contexts.

Methods
Here, we present a summary of the process to assess the evidence, including a time-restricted systematic review based on NCOC version, correlating it with current recommendations specifically looking at it from a local perspective; along with input from NCOC guidelines.
We describe the methodological challenges that exist in the developing evidence-based guidelines for an evolving pandemic. We performed a systematic review to evaluate the interventions noted in the NCOC guidelines v4 and to GRADE recommendations for pharmacological interventions for hospitalized patients. en, we compared our recommendations to WHO v7.1 COVID-19 therapeutics guidelines and subsequently invited an expert narrative review by NCOC experts specifically looking at it from a local perspective.
is study was conducted at the Center for Clinical Best Practices (CCBP), Clinical and Translational Research Incubator (CITRIC), Aga Khan University, Karachi, Pakistan, after approval from the institutional ethical review committee. e systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [3]. We aimed to review the effectiveness of pharmacological interventions included in the NCOC guidelines on mortality and length of stay in hospitalized patients with COVID-19 including evidence available until 11 th December 2020. e WHO clinical progression scale for clinical improvement (ordinal scale) was used to categorize the disease severity for each study. Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria were used to evaluate and formulate recommendations for or against and strength by considering the quality of evidence and balance between benefits and harms [4] (Table 1 and Supplementary Figure 1).
Studies in English published from 1 st November 2019 to 31 st December 2020 were included in the review. Data extraction and synthesis data extraction were performed by two independent investigators using a structured data extraction form to ensure consistency. Any disagreements were noted and resolved by further discussion with a third investigator. e extracted data are given in Tables 2-5). e quality of the final included studies was assessed according to each study design. Observational studies were assessed by the National Institute of Health Study Quality Assessment Tool [5]. Randomized control trials were evaluated by Cochrane Risk of Bias (RoB) [6]. Quasiexperimental studies were assessed by the Cochrane Effective Practice and Organization of Care (EPOC) risk of bias tool [7] (Supplementary Figures 2-9).

Results
In our systematic review, a total of 122 studies were included (Supplementary Figure 1). Data extracted from these final studies are given in Table 2 (cohort and cross-sectional studies), Table 3 (case-control studies), Table 4 (interventional studies), and Table 5 (quasiexperimental studies).
All drugs in the NCOC v4 guidelines were included, and their efficacy was assessed by evaluating length of hospitalization, mortality, and ordinal scores, and a recommendation was made based on GRADE methodology. As per the NCOC panel recommendation, we additionally included colchicine in the review and comparison of WHO, NCOC v4, and our review between drugs, as given in Table 6.

Corticosteroids.
Multiple studies have evaluated the efficacy of corticosteroids in the management of COVID-19 (Tables 2 and 5). We gave strong recommendation for the use of corticosteroids because the studies were showing early recovery in severe and critical patients; however, we gave weak recommendation for its use in noncritical patients as it did not show any positive outcomes. Most of the studies in our systematic review were observational; hence, we gave a moderate quality of evidence.
As per our systematic review, we recommend (i) For the use of corticosteroids in severe and critical patients, hospitalized COVID-19 patients. Strong recommendation, moderate-quality evidence. (ii) Against the use of corticosteroids in nonsevere patients, hospitalized COVID-19 patients. Weak recommendation, moderate-quality evidence.
WHO v7.1 dated 24/9/2021 recommended for the use of systematic corticosteroid rather than no corticosteroids in severe and critical COVID-19 patients.

Tocilizumab.
In our systematic review, mortality rates with tocilizumab therapy ranged from 6.98% to 60% (depending on patient inclusion criteria), with many studies showing a protective effect of tocilizumab with regards to mortality, especially if given intravenously (as compared to subcutaneously) and within 12 days of admission . Patients treated with tocilizumab alone were more likely to show improvement on the WHO ordinal scale (63.9% vs. 36.1%) and less likely to require ICU care (40.4% vs. 59.6%) as compared to those treated with corticosteroids in addition to tocilizumab [29]. We gave weak recommendation as the studies were showing controversial results on managing COVID-19 hospitalized adults with tocilizumab. Many of the studies were showing inconsistent results; therefore, we gave moderate certainty.
As per our systematic review, we recommend (i) For the use of tocilizumab in hospitalized COVID-19 patients. Weak recommendation, moderate-quality evidence.
WHO v7.1 recommended for the use of tocilizumab in patients with severe or critical COVID-19 infection.

Ivermectin erapy.
Due to its effectiveness in various other viral infections, ivermectin was assessed as a therapeutic agent for COVID-19 infection (Tables 2 and 4). As studies showed no major difference in mortality, we gave weak recommendation for the use of ivermectin and low quality of evidence because of insufficient evidence.
As per our systematic review, we recommend (i) Against the use of ivermectin therapy in the use of COVID-19 hospitalized patients. Weak recommendation, low-quality evidence.
WHO v7.1 recommended against the use of ivermectin in patients with COVID-19.

Antibiotics.
e macrolide azithromycin has demonstrated antiviral activity, especially in human bronchial epithelial cells where it reduces viral cell replication and causes an increase in viral-induced pattern recognition receptors. It has exhibited a synergistic effect with the drug hydroxychloroquine, and together, they decrease the production of inflammatory cytokines such as IL-1 and IL-6 [30]. We gave weak recommendation for the Studies including pregnant women.

Interventions
Observational and interventional studies describing the use of the following pharmacologic interventions for the treatment of COVID-19: (i) Steroids (dexamethasone, hydrocortisone, and prednisone methylprednisolone)                        24 Global Health, Epidemiology and Genomics 3.5. Anticoagulation erapy. As per our systematic review, the therapeutic doses of anticoagulation, i.e., nadroparin calcium (2850 IU) reduced mortality as compared to the prophylactic doses in the majority of studies [31][32][33][34]. While few studies found no difference, Billet et al. reported mortality reduction for prophylactic doses of both apixaban and enoxaparin and therapeutic doses of apixaban but not enoxaparin [35][36][37][38]. erapeutic doses of apixaban did not provide an additional mortality reduction compared to prophylactic doses. erapeutic doses of anticoagulation were shown to reduce the incidence of a venous thromboembolic event; however, both therapeutic and prophylactic doses of anticoagulation reduce in-hospital mortality compared to patients not receiving anticoagulation [31,33,35,39]. We gave strong recommendation to both therapeutic and prophylactic doses of anticoagulants, as in several studies, it has shown to reduce mortality. Many studies in our systematic review were observational, so we rated the evidence as moderate.
As per our systematic review, we recommend (i) For the use of anticoagulant therapeutic doses. erapeutic: strong recommendation, moderatequality evidence.
(ii) For the use of prophylactic dose anticoagulants to treat COVID-19 hospitalized patients. Prophylactic: strong recommendation, moderate-quality evidence.
No evidence available as per WHO v7.1.

Antivirals.
Remdesivir, an antiviral agent, has been associated with lower mortality, with one study reporting 62% lower odds of mortality and greater clinical improvement [40,41]. However, the results of other studies have not been as conclusive. Studies using the WHO ordinal scale have found weak associations between remdesivir use and improved patient outcomes [42]. With regards to other antivirals, there are conflicting data from studies. Varying WHO ordinal scale results were found for antivirals lopinavir-ritonavir. Multiple studies found no difference in mortality with the combination of ritonavir/lopinavir and remdesivir [43,44]. We gave weak recommendation for the use of remdesivir because of the inconsistent results of the studies. However, we recommend against the use of ritonavir/lopinavir use due to conflicting research evidence.
Most of the studies on remdesivir were randomized controlled trials; therefore, we rated it as high quality of evidence, while studies on ritonavir/lopinavir were mostly observational, so it has moderate certainty. As per our systematic review, we recommend

Convalescent
Plasma. Convalescent plasma initially was looked at as a possible therapy for COVID-19 infection due to its prior usefulness in other epidemic viruses. Some initial observational studies suggested the use of convalescent plasma in improving pulmonary function, decreasing adverse effects, increasing survival, and shortening hospital stay [45][46][47]. While most observational studies reported positive outcomes, RCTs have not supported convalescent plasma use. We gave weak recommendation for convalescent plasma use because RCTs have not shown significant improvement in patients' health status. As many of the studies were observational, therefore, we gave it a moderate quality of evidence.
As per our systematic review, we recommend (i) Against the use of convalescent plasma in the management of hospitalized COVID-19 patients. Weak recommendation, moderate-quality evidence.
ere are no available evidence as per WHO v7.1.

Famotidine.
While famotidine is conventionally used as an H2 receptor blocker, it also has antiviral properties [48]. Freedberg et al., in a single-centered retrospective cohort study, reported reduced mortality with famotidine use [49]. Similarly, when comparing a treatment regimen of HCQ, azathioprine, remdesivir, and corticosteroids with famotidine and those without famotidine, Mather et al. reported lower mortality in the famotidine arm (14% in the famotidine group vs. 26% in the nonfamotidine group) [50]. Despite these results, larger studies and RCTs have not yet established the role of famotidine, and therefore, we gave it low certainty. Given the minimal side effect profile of famotidine, our judgement based on data available at the time was for its use but with weak recommendation. As per our systematic review, we recommend (i) For the use of famotidine in hospitalized COVID-19 patients. Weak recommendation, low-quality evidence.
ere are no available evidence as per WHO v7.1.

Immunoglobulin erapy.
Based on the limited number of studies, a retrospective cohort study conducted by Shao et al. found IVIG therapy to reduce the 28-day mortality in critically ill patients (27% in the IVIG group vs. 53% in the non-IVIG group) [51]. Similarly, a randomized trial by Gharebaghi et al. also proved that IVIGs reduced mortality [52]. Studies have shown lower mortality; therefore, we supported its use with weak recommendation. As no significant interventional studies are supporting its use, we gave it moderate certainty. As per our systematic review, we recommend (i) For the use of immunoglobulin therapy in hospitalized COVID-19 patients. Weak recommendation, moderate-quality evidence.
ere are no available evidence as per WHO v7.1.

Colchicine.
Colchicine works by inhibiting the assembly of microtubules during mitosis by binding to tubulin inside cells and forming tight tubulin-colchicine complexes. is is its major anti-inflammatory mechanism of action [53]. In our review, only 1 single-center cohort of adult hospitalized patients with COVID-19 pneumonia and acute respiratory distress syndrome was found using colchicine 1 mg/day, investigating the association between colchicine use and improved survival in adult hospitalized patients with COVID-19 pneumonia and acute respiratory distress syndrome and found a 20.8% decrease in mortality amongst patients treated with colchicine along with other standards of care drugs [54]. Because of the insufficient research evidence, we are not recommending colchicine use and gave it a low quality of evidence.
As per our systematic review, we recommend (i) Against the use of colchicine in hospitalized COVID-19 patients. No recommendation, low-quality evidence.
ere are no available evidence as per the WHO v7.1.

Discussion
A great breadth of literature exists on the therapeutic management of hospitalized adults with COVID-19 based on disease severity, much of which have been analyzed and appraised systematically [55]. Currently, living systematic reviews provide information to guide clinical practice [56,57]. However, this approach assumes that all treatment options are available or approved in each country or region, which is not the case. In Pakistan, the Government of Pakistan has released Clinical Management Guidelines for COVID-19 Infections, based on which clinicians are expected to prescribe and treat patients [2]. While such guidelines are beneficial in which they are region-specific and keep in mind various economic factors and drug availability, they are not regularly updated due to the resources required for systematic reviews and evidence synthesis. To consolidate the pool of information to assess the effectiveness of currently approved therapeutic options for COVID-19 infection in Pakistan, we have synthesized data relating to only those treatments that are recommended in our country's guidelines using global data from 1 st November 2019 to December 31 st , 2020 [2]. Of all the drug classes analyzed, corticosteroids were found to have the most consistent effect on mortality and length of stay. e WHO clinical progression (ordinal) scale showed a reduction in mortality among patients being treated with corticosteroids. All but a few studies support the use of corticosteroids in patients hospitalized with COVID-19 [58,59]. Trials that assessed anticoagulation (especially therapeutic vs. prophylactic dosages) were also predominantly found to improve thromboembolic outcomes and mortality. However, variations in the type of anticoagulant used make it hard to recommend a single drug. Nadroparin, tinzaparin, dalteparin (LMWH), heparin, apixaban, and enoxaparin were all found to have reduced mortality [31,33,38]. Trials assessing tocilizumab supported its use to limit mortality and length of stay [13,28,29,60]. One study found that the addition of corticosteroids to tocilizumab was a significant protective factor against mortality [61].
Studies assessing remdesivir failed to show any conclusive difference in mortality and length of stays of patients with COVID-19 [42]. Of the antivirals that are being used for COVID-19, lopinavir and ritonavir have predominantly been assessed by various observational studies as well as clinical trials, both of which have been uncertain. Ribavirin alone and in combination with other antivirals (lopinavir/ ritonavir + interferon-alpha) was also shown to have minimal efficacy [62]. Data from RCTs led to recommendation against the use of convalescent plasma [63][64][65][66], which in the early days of COVID-19 was looked at as a major intervention.
Very few studies have been conducted on famotidine (an H2 receptor blocker) and IVIG. Studies have reported a significantly reduced mortality in patients being treated by famotidine or IVIG compared to control groups; however, further randomized trials and data are needed to make a concrete recommendation. Studies assessing ivermectin also report divided results and highlight the need for further studies. One of the studies assessing colchicine has reported better outcomes in adult hospitalized patients with COVID-19 pneumonia [30].
Our review is the first one to systematically review the drugs specified by the Government of Pakistan's Clinical Management Guidelines for COVID-19 Infections v4, and several other systematic reviews have been conducted to assess the efficacy of drugs used in the treatment of COVID-19, with the WHO living guidelines for pharmacological management including the most up to date data. It was thus imperative that our findings be compared and assessed against broader literature that has been published.
Our review found the lack of randomized trials to be a limitation for evidence regarding less extensively studied agents, while the more extensively investigated agents had been studied by randomized trials. Since then, newer studies, that have been conducted and included in our Global Health, Epidemiology and Genomics review, have shown tocilizumab, IVIGs, and colchicine to be effective as well. Larger trials, such as the solidarity trials, have since proven that remdesivir is not effective [67].
e urgency of information about COVID-19 infection treatment has resulted in poorly organized studies that use a variety of different outcome measures, which deters meaningful comparison between different therapeutic agents. Indeed, our review reported a wide range of outcome measures, resulting in difficulty synthesizing data.
To standardize outcome measures across studies, several international bodies worked in union and formulated a set of outcome measures, which included the WHO clinical progression scale. is is an ordinal scale, ranging from 0 (no infection) to 10 (mortality) that is especially useful in widespread diseases. e lower scores (for mild disease, which may or may not require assistance) are more subjective, and the higher scores (of severe disease requiring different levels of intervention) are likely to change based on regional practices. However, the scale is quick to use because the data required are readily available in medical records. Despite its usefulness in standardizing clinical research, the uptake of this scale has not been encouraging [68]. Our systematic review reports only 13 studies that have used this scale to report clinical progression. Gaps in reporting, with different studies grouping or failing to mention the number of patients for each score, undermines the use of a standardized scale to make sound accurate comparisons of clinical data. We recommend the use of the WHO clinical progression scale as a standard practice for studies on COVID-19 infection, with full reporting of all scores to enable comparison of study outcome measures and optimize the systematic analysis of clinical data.
ere are several limitations to this systematic review, mostly stemming from considerable heterogeneity between articles. ese include variations in participant inclusion criteria of studies, variations in outcome measures, variations in drugs used across the same class, variations in drug dosages, and variations in geographic locations and patient populations across studies. In addition, the retrospective nature of many studies, the limited sample sizes, and inadequate statistical adjustment for reported associations also adversely impact interpretability.

Conclusion
Data on pharmacological interventions to treat COVID-19 are rapidly evolving, and based on it, the recommendations have also been changed. In our systematic review, the recommendations were based on studies up till December 2021, and we have compared our recommendations with the WHO v7.1, which showed significant changes in recent treatment modalities of COVID-19 infection. Our understanding regarding the management of COVID-19 has evolved rapidly over the last two years and continues to do so. Given the urgent need to offer any therapeutic option, interim recommendations were often made based on the best available data at the time. ese data were, however, often from studies that were exploratory or not as rigorously done.
is is apparent in the disparate recommendation between the 2 guidelines and the systematic review (which is only looking at studies published during the early part of the pandemic). is also brings to light the need to continually assess the literature and be able to ready to change (previously established) therapeutic recommendations.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.