Article Text
PDF
Abstract
Background Increasing evidence has emerged for traditional, complementary and integrative medicine (TCIM) to treat COVID-19 which requires systematic summaries of the net benefits of interventions against standard care and one another. The study aims to conduct a systematic review and network meta-analysis (NMA) regarding TCIM therapies for treating mild/moderate acute COVID-19, potentially informing the WHO guideline development and clinical decision-making.
Methods and analysis We will search Embase, MEDLINE, Virtual Health Library on Traditional Complementary and Integrative Medicine, Cochrane Central Register of Controlled Trials, WHO’s International Clinical Trials Registry Platform with additional searches of Chinese and Korean literature sources for randomised controlled trials comparing TCIM with placebo, standard care, no treatment or with an alternate type of TCIM to treat COVID-19. We will limit the search dates from 2020 to 28 March 2024. Reviewers will, in duplicate, screen eligible articles, extract data and evaluate the risk of bias. We will conduct frequentist network meta-analyses for network comparisons and each outcome. We will assess the certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation approach and classify interventions from most to least effective or harmful.
Ethics and dissemination This NMA is based on the existing trials and data which is not subject to ethical approval. We will publish the NMA in a peer-reviewed journal. This may provide methodological reference and clinical evidence for TCIM on future epidemic-prone diseases.
PROSPERO registration number CRD42024517321.
- Complementary Medicine
- Network Meta-Analysis
- Protocols & guidelines
- COVID-19
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Our multidisciplinary team ensures the evidence synthesis processes have the highest methodological quality and high clinical relevance.
A group of information specialists will develop a comprehensive search strategy.
We will consider both the effect estimates and the certainty of the evidence.
We will apply Grading of Recommendations Assessment, Development and Evaluation’s approach to classify interventions from the most to the least effective, making the results easier to interpret.
Background
As of March 2024, there have been over 774 million confirmed COVID-19 cases and more than seven million deaths globally.1 Patients have shown increased interest in using traditional, complementary and integrative medicine (TCIM) globally as 170 member states of the WHO report using TCIM.2 Among 10 000 registered COVID-19 randomised controlled trials (RCTs), more than 1000 are focused on TCIM interventions.3 Thus, TCIM represents a vital category of COVID-19 treatment strategies.
According to a systematic review of observational studies, 64% (with a 95% CI of 54% to 73%) of the population in over 20 countries of North America, Europe, Asia, Africa and Australia/Oceania used TCIM in an effort to prevent or treat COVID-19.4 Developing countries experience shortages and restricted access to healthcare resources and facilities and TCIM intervention might help.5–7 Alternately, use of ineffective TCIM interventions diverts resources and may delay evidence-based care. In developed countries such as the USA, dissatisfaction with conventional healthcare services can also lead to the high usage of TCIM.8 However, up-to-date systematic summaries of the net benefit of TCIM interventions against standard care and one another are lacking.
Facing a rapidly growing body of evidence and colossal demand, clinicians and the public need trustworthy guidance regarding TCIM for COVID-19. Trustworthy guidelines for COVID-19 need methodologically rigorous and timely updated evidence summaries. Such summaries will identify evidence-based TCIM interventions to inform decision-making between clinicians and patients.
We plan to collaborate with WHO and the MAGIC Evidence Ecosystem Foundation on the Clinical Practice Guideline (CPG). We will conduct a systematic review (SR) and network meta-analysis (NMA) to examine the benefits and harms of TCIM interventions for the treatment of mild/moderate acute COVID-19. The evidence summary will potentially inform WHO guidelines and subsequently be published on the MAGICapp platform (www.magicapp.orsg).
Methods
We drafted the protocol according to Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols 20159 and registered the research in PROSPERO.
Structure and organisation
The evidence synthesis team will carry out the project with input from the NMA Steering Committee:
Evidence synthesis team
The team includes clinicians, experienced methodologists and junior team members, they will perform the following processes of evidence synthesis in sequence: (1) study identification, (2) data collection and management, (3) data analysis and (4) grading and presentation.
NMA Steering Committee
This group comprises TCIM clinical experts, systematic review and NMA methodologists and statisticians. The NMA steering group will meet regularly and make critical decisions to ensure the highest methodological quality of the evidence synthesis process and clinical relevance.
Guideline panel involvement
If the WHO decides to develop a CPG informed by the NMA, the NMA team will collaborate with a guideline development group (GDG) convened by the WHO Steering Committee. The GDG will consist of clinicians with expertise in different types of TCIM and experiences in treating COVID, methodologists and three patient partners who will be globally representative and free of any conflicts of interest. They will: (1) confirm the outcome measures, (2) direct the performance of the final subgroup analyses and meta-regression and (3) inform thresholds for adjudicating imprecision.
Patient and public involvement
The panel, if formed, will include three patient partners who have lived experience with COVID-19. They will be involved in identifying and prioritising outcomes and informing their preferences and values.
Literature search
We will search Embase, MEDLINE, Virtual Health Library on Traditional Complementary and Integrative Medicine, Cochrane Central Register of Controlled Trials, WHO’s International Clinical Trials Registry Platform and ClinicalTrials.gov. We will conduct the final search on 28 March 2024 with a date restriction from 2020 onwards applied (online supplemental file 2). We will additionally search Chinese literature sources including Wanfang, Chinese Biomedical Literature, China National Knowledge Infrastructure, Weipu Database for Chinese Technical Periodicals, Chinese Clinical Trial Registry, Chinese Medical Journal Net (preprints) and ChinaXiv (preprints); and Korean literature sources including Research Information Sharing Service, Korea Medical Database, ScienceON and Korean studies Information Service System. We will also employ the snowballing method to identify other potentially eligible studies from the reference lists of already identified RCTs or SRs.
Supplemental material
Eligibility criteria
Studies are eligible if they include patients with suspected, probable or confirmed infection with SARS-CoV-2 without age restrictions in any setting. We will include RCTs that compare any TCIM intervention with no intervention, placebo, another TCIM intervention, conventional medicine or standard care and report at least one outcome of interest. Trials can be published in press or preprints with no language restrictions. We will classify disease severity into mild/moderate COVID-19 and severe/critical COVID-19 as reported in RCTs (according to the WHO categorisation10) and this NMA will focus on mild/moderate COVID-19. Studies will be eligible if they include more than 80% of patients with mild/moderate COVID-19 or include a mixed population but reported patients with mild/moderate COVID-19 separately as a subgroup. Studies are also eligible if we could determine the severity of the included patients as mild/moderate from the description of clinical manifestations or the settings.
TCIM interventions (online supplemental file 1) include traditional medicine and practices, natural health products, mind-body therapies, physical therapies and manual therapies and further complex non-traditional medicine treatments.
Supplemental material
We will exclude vitamins, minerals, probiotics, melatonin and active ingredients derived from herbs or botanicals (eg, curcumin, quercetin) which are included in another living NMA11 on COVID-19 treatment. We will also exclude highly purified or chemically modified botanical substances and drugs without clear descriptions of compositions (eg, a self-made traditional Chinese medicine decoction without included herbs). We will exclude trials evaluating plasma therapy, external organ support, gas therapy, ventilation strategies, vaccination, psychological and educational interventions or personal protective equipment interventions alone. Quasi-randomised trials will also be excluded.
The majority of outcomes of interest will be consistent with current WHO guidelines12 13 and SRs and NMAs.11 14 If we end up with WHO guidelines, the GDG will finalise the outcomes. Reviewers will extract the outcomes measured closest to the preset time point. Candidate outcomes for the treatment of mild/moderate COVID-19 include:
Mortality (time frame: closest to 90 days).
Need for mechanical ventilation (mechanical ventilation in patients not initially mechanically ventilated, time frame: closest to 90 days).
Duration of hospitalisation.
Admission to the hospital (time frame: closest to 28 days).
Adverse effects leading to discontinuation of the intervention (time frame: closest to 28 days).
Time to all symptom resolution (as reported by the authors).
Quality of life (time frame: at the end of the treatment).
New non-SARS-CoV-2 infection.
Time to viral clearance (time until the amount of SARS-CoV2 viral particles is below the threshold set by the RCT authors for being likely no longer infectious).
Undetectable SARS-CoV-2 by PCR (time frame: closest to 7 days and not less than 4 days or more than 10 days).
The rate of conversion to severe cases (time frame: closest to 90 days).
Time to specific symptom resolution (as reported by the authors).
Prevalence of single symptom closest to 7 days (time frame: closest to 7 days and not less than 4 days or more than 10 days).
Study screening, data abstraction and risk of bias assessments
Reviewers, trained in and experienced in systematic review methods, will work in pairs, independently and in duplicate to conduct title, abstract, full-text screening, data abstraction and risk of bias assessment. Please refer to figure 1 for the study screening process.
Flow diagram of the study screening process.
For eligible studies, reviewers will use a standardised and pilot-tested extraction form to collect the information listed in Box 1.
Data Collection
First author and publication year.
Publication status (peer-reviewed publication/preprint).
Study status (completed/completed, terminated early/ongoing, interim data/ongoing, no data).
Trial registration.
Source of funding.
Geographical location(s).
Patient type (children/adults/elderly/pregnant women).
Type of care (outpatient/inpatient/intensive care).
Number of randomised participants.
Mean age.
Male proportion.
Vaccination status.
Type of virus (Alpha, Beta, Delta, Omicron, etc)
Infection status (suspected/probable, lab-confirmed).
Number of patients randomised to each group at baseline.
Number of patients analyzed in each group.
Treatment name.
Description of treatment (components, dose, frequency, route of administration and duration).
Comorbidities.
Outcomes of interest (reviewers will extract outcome data measured closest to the preset time point and according to the intention-to-treat principle).
To evaluate the risk of bias of included RCTs, reviewers will use a new, more user-friendly instrument,15 Risk of Bias Instrument for Use in Systematic Reviews—for RCTs (under user testing). Reviewers will judge the risk of bias associated with sequence generation, allocation concealment, blinding of participants, blinding of healthcare providers, blinding of outcome assessors and rate these items as ‘definitely low’, ‘probably low’, ‘probably high’ and ‘definitely high’. We will also extract the number of participants lost to follow-up in each group. We will only consider the overall bias of the trial to be high if at least one item is of definitely a high or probably high risk of bias.
All subjective steps will include calibration exercises that warrant accuracy, agreement monitoring and discussion and external adjudication when necessary.
Data synthesis and statistical analysis
Prior to statistical analysis, the NMA Steering Committee including TCIM experts who were blinded to the study results will review and categorise all the interventions for pooled pairwise analyses and into different nodes in the NMA.
For dichotomous outcomes, we will estimate the relative effects using relative risks and their 95% CIs and calculate the interventions’ absolute effects. For continuous outcomes vary largely across populations, we will estimate the relative effects using the ratio of means and their 95% CIs. We will calculate mean differences (MDs) and their 95% CIs for the post-intervention scores of other continuous outcomes. When studies use different instruments to measure general quality of life, we will transform them into the most commonly used instrument to capture their common construct.16 17 We will use continuity correction to account for zero event.18
For direct comparisons of all patient-important outcomes, when there are at least two trials, we will conduct DerSimonian-Laird random-effects19 pairwise meta-analyses. In addition, for outcomes not feasible to run NMA, we will also conduct pairwise meta-analyses for direct comparisons. For each outcome, we will check network connectivity, also ensure the availability of more trials than a number of intervention nodes or having 10 or more trials contribute data to the network. When feasible, we will conduct frequentist random-effects network meta-analyses under the assumption of common heterogeneity.
To prevent highly sparse networks, for the primary analysis, we will include treatment nodes with at least 100 participants or 20 events. Although we might not include all available TCIM interventions in the NMA, we will systematically review and report the results. Even so, when data are sparse, NMA using a random-effects model may produce counter-intuitive results (ie, although the direct and indirect estimates are coherent, the NMA may yield misleadingly wide CIs, even wider than either of them, thus decreasing the precision of NMA). This is due to the use of common heterogeneity estimates in a contrast-based model network meta-analysis and thus our analyses will use a fixed effect model.20
When the network remains largely star-shaped (that is, few, if any interventions compared with other interventions rather than to placebo or standard care), the loop-specific incoherence between direct and indirect evidence is impossible to detect statistically. We will integrate the data into a network meta-analysis under the assumption of coherence and the best estimates may come from direct evidence or indirect evidence.21 We will make decisions regarding the analysis that produces optimal estimates for individual interventions on a case-by-case basis. We will perform relevant analyses with the ‘Network’ suite in Stata (V.17.0, StataCorp, College Station, Texas, USA).
Subgroup and sensitivity analysis
We will perform a limited number of prespecified subgroup analyses to detect potential effect modifiers. We plan to explore how trials’ characteristics (low vs high risk of bias), patients’ characteristics (vaccinated vs not vaccinated), infection status (suspected or probable vs lab-confirmed) and publication status (preprints vs peer-reviewed) can influence the magnitude of the effect. We hypothesise, a priori, that treatments will yield a larger effect in trials with a high risk of bias among unvaccinated patients, among suspect or probable cases and in preprint publications. We will perform subgroup analysis only if at least two studies contribute to the subgroup. We will assess the credibility of apparent effect modification (p for interaction<0.10) using the Instrument for assessing the Credibility of Effect Modification Analyses tool22 and applying Grading of Recommendations Assessment, Development and Evaluation (GRADE) to evaluate subgroup effect estimates when effect modifications are of high or moderate credibility while evaluating overall effect estimates when effect modifications are of low or very low credibility.
If primary studies face the high risk of bias that missing outcome data may bring, we will use available participants’ outcome data as the primary analysis and perform sensitivity analysis by imputation of missing values. For binary outcomes, we will use ‘plausible worst case’, that is, for the outcome of interest of undetectable SARS-CoV-2 by PCR, we assume the incidence of outcome events in participants with missing data relative to those followed successfully is 0.2 in the treatment group; for other binary outcomes, we assume the incidence is 5. And in the control group, we assume the same event rate in participants with missing outcome data as those with available data. For continuous outcomes, we will use the worst mean among the control arms to replace the missing value in the intervention arm and the best mean among the intervention arms to replace the missing value in the control arm.23 We will use the command ‘metamiss2’ in Stata for sensitivity analysis.24
Certainty of evidence
We will use the GRADE approach to assess the certainty of evidence (CoE) for each comparison and outcome in the NMA and rate the certainty as high, moderate, low or very low.25 We will start with high certainty for direct evidence and rate it down taking into account the risk of bias, inconsistency, indirectness, imprecision and publication bias. We will evaluate heterogeneity among trials in each comparison using the I2 statistic, with I2≥50% indicating a substantial heterogeneity and also consider the context when assessing inconsistency.26 For comparisons with at least 10 studies, we will assess publication bias using Egger’s test for continuous outcomes and Harbord’s test for dichotomous outcomes;27 28 in addition, we will report potential publication bias if trials registered, completed but have no corresponding publications or reports in 1 year of trial completion.29
When intransitivity is observed, we will rate down the CoE for indirect evidence. To evaluate the transitivity assumption of indirect comparisons, we will compare distributions of potential effect modifiers (eg, age, vaccination status, type of virus, co-interventions, risk of bias) across study groups using NMA-studio web application (https://www.nmastudioapp.com).30 We will also consult our clinical experts in the Steering Committee to make sure the participants in the network share similar demographic and clinical profiles that they can be jointly randomised to any treatment included in the network.31
When direct and indirect evidence contributes the same, we rate the certainty of network estimates as the higher of the two. We take the dominance of the two when they do not contribute equally. We can further rate down the certainty of network estimate for incoherence or imprecision with judgement supported by thresholds chosen by the Steering Committee (or potentially the guideline panel). We will assess the coherence assumption for the entire network using the ‘design-by-treatment’ model (global test).32 To assess local (loop-specific) incoherence and obtain indirect estimates, we will use the node-splitting method.33 34 When incoherence is detected, we will choose the best estimates from direct or indirect evidence based on their contribution weight and CoE and if direct and indirect evidence weigh equally, we will select a network estimate and rate down its certainty. We will not rate down CoE two levels for intransitivity and inconsistency simultaneously. We will use GRADE summary of findings tables to summarise the results.
Results interpretation
To better interpret the results and support the guideline development, we will use risk differences for binary outcomes and mean differences with their 95% CIs for continuous outcomes to see if they achieve the clinically important difference deemed by the Steering Committee and potentially the guideline panels in the future. When available, we will use the baseline risk from large and methodological rigorous systematic reviews or other representative observational data. For example, for mortality, we could use CDC’s data on hospitalised patients to estimate the baseline risks35 and for mechanical ventilation or length of hospital stay, we could use the International Severe Acute Respiratory and Emerging Infection COVID-19 database to identify appropriate baseline risks.36 If we cannot identify an appropriate study, we will use the median of all included studies contributing to the standard of care node to calculate the baseline risks.
Regarding the target of certainty ratings, we will consider both the effect estimates and the certainty of the evidence and apply GRADE’s approach to classify interventions from the most to the least effective using the standard of care as the reference intervention.37
Updating
After producing the first version of the NMA, we will evaluate the feasibility of making it living. If feasible, the systematic review group will make the review living including search updating, data extraction and risk of bias assessment for eligible studies, data analysis and CoE evaluation every 6 months. If the WHO wants to use the living NMA to inform their guideline, once the WHO Steering Committee or GDG considers the emerging information likely to change practice, we will promptly respond to their requests and update the analyses. During the updating process, if the systematic review team agrees to modify analysis methods, results, etc, to accommodate new data, we will document all changes in the methods section of the updated reviews.
All updated versions will be dated and detailed changes will be recorded and then submitted to international indexes through the mechanism of corrections. The first edition and the following updates will be labelled separately.
Ethics and dissemination
Since our review is based on available evidence, ethical approval is not required. We will publish the NMA through traditional publication platforms, peer reviewed journals and potentially, the MAGICapp (https://app.magicapp.org), a novel electronic format for evidence summaries and decision aids. In addition to scientific publications, if the NMA becomes living, we will create a landing page/portal.38
Ethics statements
Patient consent for publication
Acknowledgments
The authors would like to thank Dr Janet Victoria Diaz, Dr Jamie Rylance and Julie Viry from the WHO for their helpful discussions on topics related to this research and their advice on WHO processes. We also thank Dr Kyeore Bae from National Institute of Korean Medicine (NIKOM), HAN Eunkyung from WHO Western Pacific Regional Office, Abdala, Carmen Veronica Mendes from BIREME—Pan American Health Organization and Jesus Lopez from University Hospital Zurich, Institute for Complementary and Integrative Medicine for their support in building the search.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @JasonWBusse
Contributors YZ and GG conceived the research idea. AL and YZ drafted the manuscript. KUK developed the search strategy, AL developed the Korean search strategy and JZ developed the Chinese search strategy. All authors critically revised the protocol and provided comments. All authors had final responsibility for the decision to submit for publication. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. Guarantor is YZ.
Funding This study is funded by Canadian Institutes of Health Research (Application Number: 488412); China’s ‘National Ten Thousand Talents Programme (National Top-notch Young Professionals)’ (QHe); China Academy of Chinese Medical Sciences-Special Programme for Cultivating Outstanding Young Scientific and Technological Talents (No. CI2023D004); Innovation Team and Talents Cultivation Programme of National Administration of Traditional Chinese Medicine (grant number ZYYCXTD-C-202008); Shanghai Oriental Talent Programme (LK).
Competing interests We would like to disclose that Dr. Guyatt Gordon and Dr. Thabane Lehana are listed as members of the Editorial Board of BMJ Open, as noted on their website (https://bmjopen.bmj.com/pages/editorial-board). This association has been declared in accordance with the journal’s submission policies. All other authors have no potential conflicts of interest to declare related to this submission.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.