Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
Accepted Manuscript
Ivermectin for COVID-19: addressing potential bias and medical
1
fraud
2
3
Andrew Hill1, Manya Mirchandani2, Victoria Pilkington3
4
5
1. Department of Pharmacology and Therapeutics, University of Liverpool, Liverpool, L7
6
3NY, UK
7
2. Faculty of Medicine, Imperial College London, UK
8
3. Oxford University Clinical Academic Graduate School, University of Oxford, UK
9
10
11
Corresponding author:
12
Manya Mirchandani
13
Faculty of Medicine
14
Imperial College London
15
United Kingdom
16
Email: manya.mirchandani20@imperial.ac.uk
17
18
Funding: The Rainwater Charitable Foundation
19
20
Potential Conflicts of Interest: None of the authors has declared a conflict of interest
21
22
Patient Consent Statement: All the clinical trials included in the meta-analysis were
23
approved by local ethics committees and all patients gave informed consent.
24
25
26
27
28
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
Abstract
29
30
Ivermectin has become a controversial potential medicine for COVID-19. Some early studies
31
suggested clinical benefits in treatment of infection. However, the body of evidence includes
32
studies of varying quality. Furthermore, some trials have now been identified as potentially
33
fraudulent.
34
35
We present a sub-group meta-analysis, to assess the effects of stratifying by trial quality on
36
the overall results. The stratification is based on the Cochrane Risk of Bias measures (RoB
37
2) and raw data analysis where possible.
38
39
The results suggest that the significant effect of ivermectin on survival was dependent on
40
largely poor-quality studies. According to the potentially fraudulent study (RR=0.08, 95%
41
CI=0.02,0.35), ivermectin improved survival approximately 12 times more in comparison to
42
low-risk studies (RR=0.96, 95% CI=0.56,1.66). This highlights the need for rigorous quality
43
assessments, authors to share patient-level data and efforts to avoid publication bias for
44
registered studies. These steps are vital to facilitate accurate conclusions on clinical
45
treatments.
46
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
In June 2020 ivermectin, an FDA approved anti-parasitic drug, was shown to have anti-viral
47
effects against SARS-CoV-2 in-vitro1. Following this, approximately 86 clinical trials
48
investigating ivermectin for COVID-19 have been registered globally. In late 2020, clinical
49
trials began reporting very compelling clinical benefits for ivermectin in the treatment of
50
COVID-19. From late 2020 onwards, multiple groups produced meta-analyses which
51
reported that ivermectin had a significant effect on survival, hospitalisations, clinical recovery
52
and viral clearance2, 3. Our meta-analysis was first presented in January 2021 and published
53
in July 20214. It suggested that ivermectin resulted in a significant 56% improvement in
54
survival, favourable clinical recovery, and reduced hospitalisations. Such optimistic results
55
from multiple meta-analyses have escalated public interest in using ivermectin for the
56
treatment and prevention of COVID-19, despite the WHO only recommending its use within
57
clinical trials5, 6.
58
59
However, as with all meta-analyses, a key limitation is the quality and completeness of the
60
available evidence. During our original assessment of studies, standardised Cochrane Risk
61
of Bias measures (RoB 2) had classified several studies as „high risk of bias‟7. A study by
62
Niaee et al from Iran which reported a randomised methodology, was found to have
63
significant differences in baseline characteristics across treatment arms8. This suggested
64
that the participants were not randomised appropriately, which could bias the outcomes.
65
Secondly, a study by Okumus et al from Turkey did not provide any information on allocation
66
concealment and it was unclear if the participants or investigators were blinded, which risks
67
introducing observation bias9. Lastly, a study by Hashim et al from Iraq provided insufficient
68
details on the randomisation process, lack of clarity on participants who were analysed and
69
involved unblinded assessment of a subjective outcome10.
70
71
Furthermore, some studies were then identified to be potentially fraudulent. For example, on
72
15th July 2021, a study by Elgazzar et al from Egypt was retracted from pre-print server
73
Research Square due to “ethical concerns”11. It has been reported that the data for
74
approximately 79 participants were duplicates, some deaths were recorded on dates before
75
the trial had started and instances of plagiarism were identified in the text12. Similarly, a
76
study conducted in Lebanon by Raad et al was also identified to have duplicate data for
77
multiple participants when the patient-level database was analysed in September 202113.
78
Before these inconsistencies were identified, the Elgazzar and Raad studies had been
79
included in multiple meta-analyses which suggested significant benefits for ivermectin in the
80
treatment of COVID-192, 3. In our original meta-analysis, the Raad study accounted for
81
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
11.8% of the effect of ivermectin on hospitalisation and the Elgazzar study accounted for
82
12.6% of the effect of ivermectin on survival4.
83
These instances suggest that the data available to the support the use of ivermectin for
84
COVID-19 are not reliable. In July 2021, after the potentially fraudulent studies were
85
identified, we retracted our published meta-analysis and began working on an updated
86
analysis, assessing the effects of stratifying by trial quality on the overall results. Clinical
87
trials evaluating ivermectin for the treatment of COVID-19 had been identified by systematic
88
search of 8 databases. An in-depth evaluation of study quality was conducted, in addition to
89
the standard Cochrane risk of bias tool (RoB 2) and CONSORT checklist7. Firstly, we
90
evaluated trials based on the effectiveness of their randomisation process by comparing
91
baseline characteristics across treatment arms using the chi-square test. Secondly,
92
randomisation dates were checked to ensure patients were randomised into the treatment
93
arms on similar dates. Thirdly, checks were conducted to evaluate if recruitment to treatment
94
arms was balanced at each investigational centre. Furthermore, we analysed patient-level
95
databases, where available, to check for any evidence of duplicate participants, unexpected
96
homogeneity or heterogeneity. From this, a meta-analysis was conducted with sub-groups of
97
clinical trials at different risk of bias levels.
98
99
Looking at the key survival outcome, the analysis includes 12 studies with a total of 2628
100
participants4 (Table 1). This included 4 studies at a low risk of bias, 4 with some concerns, 3
101
at a high risk of bias and 1 potentially fraudulent study. The analysis demonstrates that on
102
including all 12 studies, ivermectin results in a significant 51% increase in survival (p=0.01,
103
95% CI = 0.28-0.86) (Figure 1, Supplementary Figure 1). On excluding the potentially
104
fraudulent Elgazzar study, ivermectin results in a borderline significant 38% increase in
105
survival (p=0.05, 95% CI = 0.39-0.99) (Figure 1, Supplementary Figure 2). On excluding the
106
high risk of bias studies, ivermectin results in a non-significant 10% increase in survival
107
(p=0.66, 95% CI = 0.57-1.42) (Figure 1, Supplementary Figure 3). Lastly, on excluding
108
studies with some concerns of bias, ivermectin results in a non-significant 4% increase in
109
survival (p=0.90, 95% CI = 0.56-1.66) (Figure 1, Supplementary Figure 4). These
110
observations demonstrate that the significant effect of ivermectin on survival was dependent
111
on the inclusion of studies with a high risk of bias or potential medical fraud.
112
113
There are added challenges with clinical trials investigating COVID-19 treatments. In a rapid
114
response to COVID-19, many small-scale studies have been conducted globally for potential
115
agents. However, not all trials have reported findings. An example is a trial for Nitazoxanide
116
in Brazil with 600 participants which was completed in October 2020 but has not reported
117
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
any results yet14. Publication bias impacts meta-analyses, with positive and significant
118
results more likely to be reported. Therefore, we believe that by including publication bias, it
119
is even less likely that ivermectin will show significant benefits for COVID-19 treatment.
120
Some non-randomised trials may also be over-interpreted. For example, in a non-
121
randomised retrospective cohort study, remdesivir demonstrated an improvement in clinical
122
recovery and reduced mortality risk by 62%15. However, when evaluated in the WHO‟s
123
randomised placebo-controlled SOLIDARITY trial, remdesivir had little to no effect on
124
mortality16. Any initial promising findings from a small number of sources need to be
125
interpreted with caution, studied further and considered within the wider body of evidence.
126
127
The results from this analysis highlight the need for rigorous quality assessments when
128
evaluating clinical trials of drugs for COVID-19. Existing and widely used risk of bias
129
assessment tools are not enough. These tools provide a systematic framework for identifying
130
potential key sources of bias in a trial‟s internal methodology but work on the fundamental
131
assumption that a published study is reporting accurate and complete findings. They allow
132
reviewers to make judgements on the assumption that basic standard procedure is followed,
133
the data are real, and that no information is being intentionally hidden.
134
135
With cases of potential medical fraud now identified, it is essential that access to patient-
136
level databases is provided. If authors fail to provide these data, the study should be
137
considered with a higher index of suspicion. Additionally, it should be mandatory that all
138
registered trials report their findings. We understand that these are substantial changes to
139
established procedures. However, the failure to recognise the potentially fraudulent studies,
140
which led to multiple meta-analyses suggesting significant benefits of ivermectin for COVID-
141
19, indicates that the tools currently used to evaluate the quality of clinical trials are
142
insufficient. These events warrant our stringent recommendations.
143
144
145
Acknowledgements
146
We would like to thank Gideon Meyerowitz-Katz, Kyle A. Sheldrick and Jack M. Lawrence for
147
their help with the quality assessments.
148
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
REFERENCES
149
150
1) Caly, L., Druce, J., Catton, M., Jans, D., & Wagstaff, K. (2020). The FDA-approved drug
151
ivermectin inhibits the replication of SARS-CoV-2 in vitro. Antiviral Research, 178, 104787.
152
doi: 10.1016/j.antiviral.2020.104787
153
154
2) Bryant A, Lawrie TA, Dowswell T, Fordham EJ, Mitchell S, Hill SR, et al. Ivermectin for
155
Prevention and Treatment of COVID-19 Infection: A Systematic Review, Meta-analysis, and
156
Trial Sequential Analysis to Inform Clinical Guidelines. American Journal of Therapeutics.
157
2021;28(4).
158
159
3) Kory P, Meduri GU, Varon J, Iglesias J, Marik PE. Review of the Emerging Evidence
160
Demonstrating the Efficacy of Ivermectin in the Prophylaxis and Treatment of COVID-19.
161
American Journal of Therapeutics. 2021;28(3).
162
163
4) Hill, A., Garratt, A., Levi, J., Falconer, J., Ellis, L., & McCann, K. et al. (2021). Meta-
164
analysis of randomized trials of ivermectin to treat SARS-CoV-2 infection. Open Forum
165
Infectious Diseases. doi: 10.1093/ofid/ofab358
166
167
5) Desperation, misinformation: how the ivermectin craze spread across the world. (2021).
168
Retrieved 15 October 2021, from
169
https://www.theguardian.com/world/2021/sep/24/ivermectin-covid-peru-misinformation
170
171
6) WHO. Therapeutics and COVID-19: Living Guideline 2021. Retrieved 13 October 2021.
172
Available from: https://www.who.int/publications/i/item/WHO-2019-nCoV-therapeutics-
173
2021.2.
174
175
7) Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng
176
H-Y, Corbett MS, Eldridge SM, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni
177
P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart
178
LA, Tilling K, White IR, Whiting PF, Higgins JPT. RoB 2: a revised tool for assessing risk of
179
bias in randomised trials. BMJ 2019; 366: l4898.
180
181
8) Gheibi, N., Shakhsi Niaee, M., Namdar, P., Allami, A., Zolghadr, L., & Javadi, A. et al.
182
(2021). Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: A
183
randomized multi-center clinical trial. Asian Pacific Journal Of Tropical Medicine, 14(6), 266.
184
doi: 10.4103/1995-7645.318304
185
186
9) Okumuş, N., Demirtürk, N., Çetinkaya, R., Güner, R., Avcı, İ., & Orhan, S. et al. (2021).
187
Evaluation of the effectiveness and safety of adding ivermectin to treatment in severe
188
COVID-19 patients. BMC Infectious Diseases, 21(1). doi: 10.1186/s12879-021-06104-9
189
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
190
10) Hashim, H., Maulood, M., Rasheed, A., Fatak, D., Kabah, K., & Abdulamir, A. (2020).
191
Controlled randomized clinical trial on using Ivermectin with Doxycycline for treating COVID-
192
19 patients in Baghdad, Iraq. doi: 10.1101/2020.10.26.20219345
193
194
11) Elgazzar Aea. Efficacy and Safety of Ivermectin for Treatment and prophylaxis of
195
COVID-19 Pandemic [retracted]. Research Square. 2021.
196
197
12) Reardon S. Flawed ivermectin preprint highlights challenges of COVID drug studies.
198
Nature. 2021; 596, 173-174
199
200
13) Samaha AA, Mouawia H, Fawaz M, Hassan H, Salami A, Bazzal AA, et al. Effects of a
201
Single Dose of Ivermectin on Viral and Clinical Outcomes in Asymptomatic SARS-CoV-2
202
Infected Subjects: A Pilot Clinical Trial in Lebanon. Viruses. 2021;13(6).
203
204
14) Nitazoxanide Therapy for Patients With COVID-19 Pneumonia - Full Text View -
205
ClinicalTrials.gov. (2021). Retrieved 13 October 2021, from
206
https://clinicaltrials.gov/ct2/show/NCT04561219?cond=nitazoxanide&draw=2&rank=6
207
208
15) Gilead. Gilead Presents Additional Data on Investigational Antiviral Remdesivir for the
209
Treatment of COVID-19 2020. Retrieved 13 October 2021. Available from:
210
https://www.gilead.com/news-and-press/press-room/press-releases/2020/7/gilead-presents-
211
additional-data-on-investigational-antiviral-remdesivir-for-the-treatment-of-covid-19.
212
213
16) WHO. Repurposed Antiviral Drugs for Covid-19 — Interim WHO Solidarity Trial Results.
214
New England Journal of Medicine. 2020;384(6):497-511.
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
230
Study
Risk of bias level
Sample size
Ivermectin arm
Control arm
Lopez-Medina et al
Low risk
398
0/200
1/198
Fonseca et al
Low risk
168
12/53
25/115
Zoni et al
Low risk
501
4/250
3/251
Kirti et al
Low risk
112
0/55
4/57
Rezai et al
Some concerns
69
1/35
0/34
Abd-Elsalam et al
Some concerns
164
3/82
4/82
Gonzalez et al
Some concerns
73
5/36
6/37
Mahmud et al
Some concerns
363
0/183
3/180
Niaee et al
High risk
180
4/120
11/60
Hashim et al
High risk
140
2/70
6/70
Okumus et al
High risk
60
6/30
9/30
Elgazzar et al
Apparent fraud
400
2/200
24/200
Total
2628
39/1314
96/1314
Table 1: Studies included in the survival analysis4
231
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022
Accepted Manuscript
Figure 1: Survival effects of ivermectin?
Downloaded from https://academic.oup.com/ofid/advance-article/doi/10.1093/ofid/ofab645/6509922 by guest on 18 January 2022