Content uploaded by Jessica Rose
Author content
All content in this area was uploaded by Jessica Rose on Apr 21, 2023
Content may be subject to copyright.
87
Science, Public Health Policy,
and the Law
Volume 3:100–129
October, 2021
Clinical and Translational
Research
An Institute for Pure
and Applied Knowledge (IPAK)
Public Health Policy
Initiative (PHPI)
Critical Appraisal of VAERS Pharmacovigilance: Is
the U.S. Vaccine Adverse Events Reporting System
(VAERS) a Functioning Pharmacovigilance System?
Jessica Rose, PhD, MSc, BSc
The Institute for Pure and Applied Knowledge
“Patterns of adverse events, or an unusually high number of adverse events reported after a particular
vaccine, are called ‘signals.’ If a signal is identified through VAERS, scientist[s] may conduct further
studies to find out if the signal represents an actual risk.”
CDC on Vaccine Safety
Abstract
Following the initiation of the global rollout and administration of the COVID-19 vaccines1,2 on
December 17, 2020, in the United States, hundreds of thousands of individuals have reported Adverse
Events (AEs) using the Vaccine Adverse Events Reports System (VAERS). To date, approximately 50%
of the population of the United States have received 2 doses of the COVID-19 products with 427,831
AEs reported into VAERS as of August 6th, 2021.
Pharmacovigilance (PV) is the process of collecting, monitoring, and evaluating AEs for safety signals
to reduce harm to the public in the context of pharmaceutical and biological agents. Many of the issues
with VAERS are becoming well known – especially with regards to reporting and recording of data – in
light of the extensive use of this system this year, challenging its functionality as a pharmacovigilance
system.
This appraisal assesses three issues that respond to the question of VAERS pharmacovigilance by
analyzing VAERS data: 1. deleted reports, 2. delayed entry of reports and 3. recoding of Medical
Dictionary for Regulatory Activities (MedDRA) terms from severe to mild. The most recently updated
publicly available VAERS dataset was found to have N=1516 (0.4%) VAERS IDs removed (“missing”).
1
The Brand Name: Pfizer-BioNTech COVID-19 Vaccine, the Previous Name: BNT162b2 or the Company Name:
Pfizer Inc. and BioNTech SE. can be used in the case of the Pfizer/BioNTech COVID-19 products. The Brand
Name: mRNA-1273 and/or Company Name: Moderna, Inc. can be used in the case of the Moderna COVID-19
products.
2
mRNA biologicals are not true vaccines. True vaccines undergo time-dependent testing protocols to ensure safety
and efficacy, typically enduring between 10 and 15 years. True vaccines are a preparation of a weakened or killed
pathogen, such as a bacterium or virus, or of a portion of the pathogen’s structure that, upon administration to an
individual, stimulates antibody production or cellular immunity against the pathogen but is incapable of causing
severe infection. The mRNA biologicals do not satisfy either these requirements and as such are more akin to
experimental treatments than vaccines.
101
Of this missing data, 13% represented death, 11% represented COVID-19 and 63% represented Severe
Adverse Events (SAEs). Of these missing death data, only 59% represented redundancies – re-assigned
new VAERS IDs – the remainder were unaccounted for.
A lag time between onset of AEs and entry of AEs into the VAERS public database was discovered,
and it appears to depend on the AE type. For example, in the case of COVID-19 breakthrough cases,
approximately mid-May, 4100 (38% of total) reports were retroactively added approximately 8.5 weeks
following the original onset date. SAEs were not found to be downgraded to mild AEs (MAEs) for a
tested cohort within 10 selected updates.
VAERS is designed to reveal potential early-warning risk signals from data, but if these signals are not
detectable as they are received, then they are not useful as warnings. Considering the relevance of safety
concerns in the face of the large numbers of AEs being reported into the VAERS system in the context
of COVID-19 products, it is essential that the VAERS system be carefully and meticulously maintained.
Despite the emergence of the Standard Operating Procedures (SOP) for COVID-19, VAERS is lacking
in transparency and efficiency as a PV system, and it requires amendment or replacement.
Copyright © The Author — Published Under the Creative Commons License
Share/Alike (see https://creative commons.org/licenses/)
Correspondence: jrose@ipaknowledge.org
Keywords
COVID-19; Vaccine Adverse Events Reports System (VAERS); Adverse Events (AEs); Severe Adverse Events
(SAEs); Mild Adverse Events (MAEs); VAERS Wayback Machine; Standard Operating Procedures (SOP);
Medical Dictionary for Regulatory Activities (MedDRA); Pharmacovigilance (PV)
Contents
1
Background
101
2
Methods
103
3
Results
105
4
Discussion
113
5
Conclusion
115
6
References
116
7
Supplementary Materials
123
8
Author statements
129
1
Background
Pharmacovigilance is the process of collecting,
monitoring, and evaluating AEs for safety signals
to reduce harm and promote safety to the public in
the context of pharmaceutical and biological agents
[1,2]. There are a number of organizations and
agencies that exist to ensure pharmacovigilance as
part of regulation of biological products from
conception to administration into humans for use.
The Center for Biologics Evaluation and Research
(CBER), as an example, actively participates in
international pharmacovigilance efforts under the
umbrella of the Food and Drug Administration
(FDA) and the Department of Human Health
Services (DHHS) [3]. International regulatory
organizations such as the World Health
Organization (WHO), the Pan American Health
Organization (PAHO) and the World Intellectual
Property Organization (WIPO) also function to
ensure pharmacovigilance in biologicals and serve
as sources of guidance pertaining to pharmaco-
vigilance efforts. In addition, individual countries
have their own regulatory authorities, such as the
Medicines & Healthcare products Regulatory
Agency (MHRA) of the United Kingdom (U.K.),
responsible for rule and regulation enforcement and
the issuance of guidelines to ensure pharmaco-
vigilance in the development and administration of
biological products. The U.K. ‘Coronavirus Yellow
Sci, Pub Health Pol, & Law Critical Appraisal of VAERS Pharmacovigilance — Oct. 2021
102
Sci, Pub Health Pol, & Law Critical Appraisal of VAERS Pharmacovigilance — Oct. 2021
Card’ reporting site allows collection of AE data
monitored by the MHRA.
The U.S. FDA and Centers for Disease Control
and Prevention (CDC) created and implemented the
Vaccine Adverse Event Reporting System
(VAERS) in 1990 to receive reports about AEs that
may be associated with biological products such as
vaccines.
3
Most vaccine AE reports in VAERS
concern relatively minor events, such as injection
site pain. Other reports describe serious events,
such as hospitalizations, life-threatening illnesses,
or deaths [4,5,6,7,8]. The reports of serious events
are of greatest concern and are meant to receive the
most scrutiny by VAERS staff and healthcare
professionals. The primary purpose of the database
is as a pharmacovigilance tool – to serve as an early
warning or signaling system for AEs not detected
during pre-market testing. The National Childhood
Vaccine Injury Act of 1986 (NCVIA) requires
health care providers and vaccine manufacturers to
report AEs to the DHHS following the
administration of vaccines outlined in the Act
[4,5,6,7]. Reported AEs, as part of the VAERS
system, represent a fraction of the actual number of
AE incidents, so the numbers reported herein are
likely far lower than actual numbers [6,7,9].
VAERS reports can be made by nurse practitioners,
general practitioners, or family members, which
can result in duplicate reports being made. As part
of the VAERS Standard Operating Procedures for
COVID-19 (SOP)
4
published on January 29th,
2021, the CDC and the FDA are meant to perform
routine VAERS surveillance to identify potential
emergent safety concerns in the context of COVID-
3
VAERS has benefits of the PREP Act – while vaccine manufacturers are shielded from liability, and vaccine
proponents tout VAERS as an example of active PV, VAERS users must acknowledge the data cannot be used to
establish causality.
4
Vaccine Adverse Event Reporting System (VAERS), Standard Operating Procedures for COVID-19 (as of 29
January 2021), VAERS Team: Immunization Safety Office, Division of Healthcare Quality Promotion National
Center for Emerging and Zoonotic Infectious Diseases and Centers for Disease Control and Prevention.
5
NIA Adverse Event and Serious Adverse Event Guidelines (2018).
https://www.nia.nih.gov/sites/default/files/2018-09/nia-ae-and-sae-guidelines-2018.pdf
19 injectable products [5,6,7,10,11,12]. Accordingly,
VAERS reports are received, processed, and
managed by trained CDC contractors. The VAERS
reports are received online for subsequent review,
and symptoms and diagnoses are assigned
MedDRA standard codes. Additional information,
including hospital records and autopsy reports, will
be requested by these trained staff when appropriate,
as outlined in the SOP. Reports are often changed
or deleted. For example, in the case where a person
successfully files a report using the VAERS system
and subsequently dies, they are, in some cases,
assigned a new VAERS ID number, unlinking their
reported AEs and death records. In addition, as the
AEs may become more enumerable in an
individual, multiple changes can be made to their
VAERS report under the same VAERS ID number
or, as indicated, under a different VAERS ID
number if they die.
An Adverse Event (AE) is defined as any
untoward or unfavorable medical occurrence in a
human study participant, including any abnormal
physical exam or laboratory finding, symptom, or
disease temporally associated with the participants’
involvement in the research, whether or not
considered related to participation in the research.
Based on the Code of Federal Regulations, a
Serious or Severe Adverse Event (SAE)
5
is defined
as any adverse event that results in death, is life
threatening, or places the participant at immediate
risk of death from the event as it occurred, requires
or prolongs hospitalization, causes persistent or
significant disability or incapacity, results in
congenital anomalies or birth defects, or is another
103
Sci, Pub Health Pol, & Law Critical Appraisal of VAERS Pharmacovigilance — Oct. 2021
condition which investigators judge to represent
significant hazards.
6
The VAERS handbook states
that approximately 15% of reported AEs are
classified as severe [4]. Nowhere in the VAERS
handbook or on the website published by the
CDC/FDA is there mention of deleted data or
transparent description of the processes and criteria
used for record deletion. The only reference I could
find to legitimate removal of data, from
WONDER’s ‘Reporting Issues’ section, claims that
‘Duplicate event reports and/or reports determined
to be false are removed from VAERS’.
7
A Wayback Machine
8
is an initiative of the
Internet Archive, a 501(c)(3) non-profit, building a
digital library of Internet sites and other cultural
artifacts in digital form. The VAERS Wayback
Machine
9
therefore allows an examination of the
VAERS government data input each week. The
U.S. Government publishes a new version of its
VAERS database weekly and VAERS IDs can be
changed or even deleted without documentation of
edits. The VAERS Wayback Machine provides a
way to trace and track deleted files based on
matches in field entries between VAERS ID
versions.
10
2
Methods
General methodology and descriptive statistics
To analyze the VAERS data sets, R was used. (R: a
language and environment for statistical
computing.) VAERS data are accessed through the
CDC Wide-ranging Online Data for Epidemiologic
Research (WONDER) system. The VAERS data
are available for download
11
in three separate
6
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.cfm?
7
VAERS data can be accessed through the CDC Wide-ranging Online Data for Epidemiologic Research
(WONDER) system. https://wonder.cdc.gov/vaers.html
8
https://web.archive.org/
9
https://medalerts.org/vaersdb/wayback/
10
https://www.cdc.gov/vaccinesafety/ensuringsafety/monitoring/vaers/index.html
11
https://vaers.hhs.gov/data/datasets
comma-separated values (csv) files representing (i)
general data for each report; (ii) the reported AEs or
‘symptoms’; and (iii) vaccine data for each report,
including vaccine manufacturer and lot number.
The VAERS dataset is updated weekly. Upon
individual reporting of vaccine side effects or AEs,
a VAERS ID number is provided to the individual
to preserve confidentiality, and a detailed
description of the AEs are transcribed along with
the individual’s age, residence by state, past
medical history, allergies and gender, and many
other details. In addition, the vaccine lot number,
place of vaccination and manufacturer details are
included in the report.
The VAERS ID was used as a linking variable to
merge the three csv files. Data was filtered
according to vaccine type (reports made only for
COVID-19), and all variables were retained,
including VAERS ID, AEs, age, gender, state,
vaccination date, date of death, incident of death,
dose series, treatment lot number, treatment
manufacturer, hospitalizations, emergency depart-
ment visits, disabilities, life threatening AEs, birth
defects and onset date of AEs. Deaths are
categorized according to whether or not the
individual had been marked as ‘DIED’. Erroneous
labelling is an issue in VAERS, for example, when
‘Death’ is an AE and yet the ‘DIED’ column is
marked ‘NA’ or ‘not applicable’, thus the dataframe
was checked and corrected for inconsistencies in
the ‘DIED’ column vector. For the purposes of this
analysis, deaths according to VAERS classification
by ‘DIED’ plus these corrected cases of
misclassification are reported here and used in the
analysis. The grouped AE categories hospitalizat-
ions and emergency doctor visits were created by
104
Sci, Pub Health Pol, & Law Critical Appraisal of VAERS Pharmacovigilance — Oct. 2021
selecting ‘Y’ in the respective column vectors,
while the cardiovascular, neurological and
immunological groups were created by selecting
keywords indicative of a respective medical issue.
The SAEs were classified according to whether the
individual succumbed to death, was hospitalized,
was admitted to the ER, experienced a disability or
a life-threatening AE, or if a birth defect ensued.
It should be noted at this point that anyone using
the VAERS WONDER system will not see the
same counts that are described in this analysis, since
hospitalizations, ER visits and all SAEs counts
were calculated by counting the ‘Y’ entries in the
respective fields in the merged file. The difference
between the counts in this analysis and counts from
a WONDER query are simply due to the effect of
losing field entries by merging the files. If one uses
the files available for download from the VAERS
website with the aim of comprehensive analysis of
the full range of data, the 3 csv files must be
merged. In order to know what ‘SYMPTOMS’ an
individual succumbed to prior to death, for
example, or to know what injectable product they
were given, it is necessary to merge the DATA file
with the SYMPTOM file and the VAX file. It is also
vital to omit redundancies in VAERS IDs – if not
done, this could lead to excess numbers in absolute
counts. The downside to the merge is loss of data
due to incomplete field entries; however, it is
important to note that the merge counts are under-
approximations, yet still prove the points made
herein.
Deleted data were isolated and aggregated by
using anti-join iterations in R on sequential
dataframes. Anti-join returns the rows of the first
dataframe that are not matched in a second
dataframe. This was done iteratively for all
sequential dataframes, and the unmatched data were
aggregated and put into a new file entitled ‘missing
12
Onset Date (ONSET_DATE): The date of the onset of adverse event symptoms associated with the vaccination
as recorded in the specified field of the form.
13
Today’s date (TODAYS_DATE): Date Form Completed.
data’. The collective missing data file was
subsequently filtered for duplicates to ensure that
redundancies were omitted.
A missing VAERS ID can be missing due to
having been removed because it is redundant, or for
reasons yet unknown. The former entries are re-
assigned a new VAERS ID and are traceable by
matching fields in column vectors of dataframes.
The latter are missing due to unknown reasons. To
discern between redundant and deleted VAERS
IDs, deleted data were cross-referenced by
matching fields for relevant selected variables in the
most recently updated publicly available dataset.
This was done only for the deleted death data, since
it is a time-consuming exercise. The matching
algorithm was as follows: match age, state, and
gender followed by vaccine lot if available, onset,
vaccine and death dates followed by allergies,
medications, and any other unique identifiers of the
individual. If a match was found, the newly
assigned VAERS ID was recorded alongside the old
VAERS ID in a new file. If a match was not found,
then the VAERS ID was deemed to have been
deleted from the database.
Two methods were used to investigate temporal
lags in data entry. The first method involved using
only the most recently updated publicly available
dataset. Assessment of temporal differences in data
entry was done by calculating the difference in the
number of days between the onset date
(ONSET_DATE)
12
and the date that the AE was
entered into the VAERS database (TODAY’S_
DATE).
13
The second method involved comparing
the data from the weekly updates to the most
recently updated file. Each week, a new set of data
is available for download from the VAERS website,
as mentioned previously. As an example of how the
data sets were compared, consider the first and the
last VAERS datasets available for download in
105
Sci, Pub Health Pol, & Law Critical Appraisal of VAERS Pharmacovigilance — Oct. 2021
2021. According to a reference variable, such as the
ONSET_DATE, these two datasets should both and
equally capture all AEs submitted to VAERS from
January 1st through January 7th, 2021, since the
first available dataset would comprise the first week
of data. If any two datasets do not equally capture
all AEs, then this discrepancy would warrant
explanation. A feasible explanation for a non-match
in the number of VAERS IDs per ONSET_DATE
entries reported would be retroactive addition of
reports to the system due to a backlog.
The incidence of SAE downgrade to MAE was
assessed by choosing 10 update files, calculating
the SAE and MAEs, and subsequently comparing
them to original counts for SAE and MAE in the
original files. This was done using the semi-join
function in R.
Statistical Testing
Statistical analysis was done using the Student’s t-
Test to determine statistically significant differences
between AE types in the deleted data file. Skewing
in distribution of data was tested using Pearson’s
Skewness Index, I, which is defined as I = (mean-
mode)/standard deviation. The data set is
considered to be significantly skewed if |I|≥1.
3
Results
3.1 Historical pharmacovigilance of VAERS
and other safety monitoring systems
VAERS and other safety monitoring systems have
been useful for pharmacovigilance in the past. In
2010, rotavirus vaccines licensed in the U.S were
found to contain Porcine circovirus (PCV) type 1
and were subsequently suspended. On 22 March,
2010, the FDA issued a statement recommending
that clinicians and public health professionals in the
United States temporarily suspend the use of
Rotarix [13,14,15]. In 2009, an increased risk of
narcolepsy was found following vaccination with a
monovalent H1N1 influenza vaccine that was used
in several European countries during the H1N1
influenza pandemic [15,16,17]. Between 2005 and
2008, a meningococcal vaccine was suspected to
cause Guillain-Barré Syndrome (GBS) [15,18]. In
1998, a vaccine designed to prevent rotavirus
gastroenteritis was associated with childhood
intussusception after being vaccinated [15,19–29].
Also in 1998, a hepatitis B vaccine product was
linked to multiple sclerosis (MS) [15,30].
Pharmacovigilance has functioned in the context of
COVID-19 VAERS data with regards to
myocarditis, resulting in a COVID-19 vaccine
safety update by the Advisory Committee on
Immunization Practices (ACIP, June 23rd, 2021) by
Tom Shimabukuro. The report did not result in any
changes to the rollout despite the danger signal
having arisen [31].
To date, 50% of the total US population has
received 2 doses of COVID-19 products,
14
with
427,831 AEs reported as of August 6th, 2021. These
numbers are off the scale with regards to numbers
associated with vaccine rollouts when compared to
previous years. Even more atypical are the numbers
of deaths reported in the context of the COVID-19
products. Figure 1 shows the total VAERS reports
from data and total VAERS-reported death counts
per year for the past 10 years up to and including
the VAERS update on August 6th, 2021. Both the
absolute numbers of total AEs and those of deaths
per year dramatically outnumber the absolute
numbers recorded in previous years. To date, there
are 6639 (1.6% of all AEs) deaths in the VAERS
database. Normalization to fully injected
populations were done and compared with
INFLUENZA vaccine data for past years and it was
14
https://usafacts.org/visualizations/covid-vaccine-tracker-states/
106
Sci, Pub Health Pol, & Law Critical Appraisal of VAERS Pharmacovigilance — Oct. 2021
found that the increase in AEs is not due simply due
to an increase in injections [32].
As part of an ongoing analysis [8], VAERS data
are being monitored according to weekly updates.
Figure 2 shows the total AE count (up to and
including the August 6th, 2021, VAERS update) by
age group alongside the SAE data by age group
(according to CDC age group classifications). The
distribution in both cases is symmetric and
unimodal, not skewed toward any particular age
group, potentially meaning that there is no
particular age group with lesser chance of
succumbing to an AE or, more importantly, an
SAE. Of the SAEs, there are 6,639 deaths, 26,402
hospitalizations, 59,061 ER visits, 7,423 life-
threatening events, 6,861 disabled and 258 birth
defects reported.
Female reproductive issues (FRIs) and AEs in
children aged 12–18 years are on the rise. There are
currently 6,398 total FRIs and 18,021 AEs reported
in young children aged 12 through 18. These
children represent 4.2% of the total VAERS data
and 12.9% of all cardiovascular AEs. It should be
highlighted that the rollout has only just begun
recently for children in these young demographics.
Figure 3 shows histograms for the FRIs (left) and
Figure 1: Bar plots showing the number of VAERS reports (left) and reported deaths (right) per year for
the past decade. (2021 is partial data set.)
Figure 2: Histogram plots showing distributions of the AEs of the total VAERS ID count (left) and for
SAEs (right).
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
107
for the children (right) with respect to age in years.
Most reports within the children aged 12–18 were
made for 17-year-olds.
3.2 Missing data
To date (August 6th, 2021), 1,516 VAERS IDs are
missing from the most recently updated publicly
available VAERS database. This represents 0.4% of
the total VAERS IDs. For each of the 28 updates,
one anti-join iteration was performed between
sequential updates. For each anti-join iteration, of
which there are currently 27, the extracted missing
data counts are as follows: 10, 13, 20, 20, 4, 12, 30,
18, 41, 14, 25, 24, 45, 72, 89, 77, 69, 102, 53, 115,
89, 167, 95, 63, 62, 87 and 101. That is, between the
first update and the second, 10 VAERS IDs are
missing; between the second and third, 13 VAERS
IDs are missing, and so on up to the second-last and
the most recent update where 101 VAERS IDs are
missing. Figure 4 shows the distribution of the
missing data according to age groups for the entire
missing data set (left) and for the SAEs within the
set (right). The missing data are distributed in a
symmetric and unimodal way with regards to age
groups and are not skewed toward any group in a
statistically significant way (I=-0.2) when
compared to the dataset without removals.
Interestingly, when the data are not filtered by
age group, 63% of all missing data reports qualify
Figure 3: Histogram plots showing the distributions of female reproductive issue AEs and AEs in children
aged 12–18 years old from the VAERS dataset according to age group (left) and age in years (right).
Figure 4: Histogram plots showing the distributions of the missing data of the total AE counts (left) and
for SAEs (right) from the VAERS dataset according to age group.
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
108
as Severe AEs, and this represents 1.2% of the total
SAEs reported to VAERS. When the data are
filtered by age group, this percentage becomes 81%,
as shown in Figure 4. The missing SAE data are
distributed in a symmetric and unimodal way with
regards to age groups and are not skewed toward
any group in a statistically significant way (I=-0.4).
Of the total missing VAERS ID data set, 41% of
the missing IDs involved hospitalizations and 37%
involved emergency room visits (data not shown).
Histograms of these two categories do not show any
statistically significant skewing toward any
particular age group (I=0.1 and I=-0.1, respectively;
not shown).
Individuals who succumb to and are diagnosed
with COVID-19 post-injection, also known as
breakthrough events, comprise 11% of the total
missing data (1.4% of total VAERS IDs). It is very
strange to report that 70% of the age data contains
an “NA” entry in the “AGE_YRS” field and thus
age-grouped data analysis is not tenable here. FRIs
comprise 0.8% of the missing VAERS IDs (0.2%
of total FRIs reported to VAERS).
3.2.1 Death data comprises 13% of missing data
Although the absolute number of missing VAERS
IDs may not be high, of this small subset of deleted
data, 13% of total missing AEs are deaths. The total
number of deaths is 199 and in each sequential
iteration of the anti-joining of the datasets, death
remained at the highest or near highest frequency
for missing AEs in each “SYMPTOM” list for the
extracted missing data set, save for SYMPTOM
column 5, which rarely contains the primary or
most prevalent AE reported per individual. For
example, of the 5 SYMPTOM column variables
representative of the reported AEs, SYMPTOM
column 1 primarily contains the most prevalent AE
listed and has ‘COVID-19’ as the #1 most frequently
occurring missing report (22%) with ‘Death’ at #2
(15%). This missing death data comprises 3% of the
total VAERS death reports.
Figure 5: A histogram plot showing distribution
of missing death data according to age group
Figure 5 shows that the distribution of deleted
death data is asymmetric, unimodal and not skewed
in a statistically significantly way toward any
specific age group in this data set (Figure 7 (I)=0.7).
Of the missing death data, 15% of reports were
made within 24 hours and 28% of reports were
made within 48 hours indicating a clustering of
reports in very close temporal proximity to the
injection.
3.3 Redundancy deletions versus deletions for
unknown reasons in death reports
There are 199 deleted death entries to date from the
VAERS database and 214 deleted death entries to
date collected from the VAERS Wayback Machine.
The discrepancy of 15 deleted deaths, which
accounts for 3% of all reported deaths, arises from
deletions of individuals in a ‘foreign location’ that
are not included on the publicly available Domestic
dataset. The deleted death data list can be found in
the Supplementary materials. Deletions of
redundant entries are marked by NA in the ‘True
deletions’ column and the accompanying new
VAERS IDs are listed. Deletions due to unknown
reasons are marked by TRUE value in the ‘True
deletions’ column. Of the total list, 59% were found
to be redundant entries and 41% of the entries were
true deletions. For the remaining 1317 non-death-
related AEs, a cross-reference search would need to
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
109
be completed in future work to discover what
percentage of total missing AEs are true deletions.
3.4 Unexplained lag in data input
An anomaly in the data pertaining to data entry
times when compared to onset of AE dates can be
seen when total AE counts reported in the most
recently updated publicly-available VAERS dataset
(updated August 6th, 2021) are compared with total
AE counts as per VAERS weekly updates. To date,
there are 28 sets of data, and discrepancies can be
found between the files from update to update. This
would not necessarily be perceived by a data
analyst if they were simply looking at the data from
the most recently uploaded data to the VAERS
system. One would only notice this discrepancy if
simultaneously analyzing the individual sets as
compared with the most recently updated set by
update date. If the VAERS system was functioning
as a pharmacovigilance system and in fact passive,
these data sets would be expected to follow the
same trajectory. Evidently, there are two
trajectories, and they are not similar quantitatively
or qualitatively.
Figure 6 (left) shows the number of deaths for
each specific update date per week. For example,
the first row of bars with x-axis marker ‘1’ shows
the number of deaths for each of the updates
according to weeks 1–27 (01/30/21–07/30/21). A
closer look (examining only weeks 1–12 for clarity)
at Figure 6 (right) reveals that the number of deaths
were essentially equal for the first 12 updates for
week 1. By week 12, this number started to change
with respect to week-by-week calculations of death
counts. If we observe the slope of the difference in
absolute number in the data per update date, it is
increasing quite consistently as the week number
increases. This is precisely what we would expect
to see if data were being retroactively added. The
inconsistency is the increasing slope that emerges.
It should not be increasing – not even remotely. The
only increase we would expect to see is a grouped
increase over a week. Absolute numbers should not
change per week with respect to weekly data
already entered. Thus, if data are being retroactively
added, then we would see changes reflected per
week as shown in the red rectangle on the right in
Figure 6 (right).
Another way to visualize this phenomenon is
using a heatmap. Figure 7 is a correlation plot
illustrating the number of deaths per week for death
week versus the week of entry into the VAERS
Figure 6: Bar plots showing the discrepancies in cumulative data by slope of increase at the beginning of
the data versus slope of decrease at the end (current update)
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
110
Figure 7: Heatmap showing the delayed death
data entries where n is the number of deaths per
intersection tile
database. Any entry that is not on the diagonal is an
entry that was not entered on the week that the
person died. 21 tiles (42%) representing n>1 deaths
indicates that many entries were entered well after
the death date. In one case, the AE was entered 77
days post death. This is clear evidence of death data
being retroactively added. Considering that death
certificates can take time to be processed, it is to be
expected that some death entries to VAERS would
occur quite temporally distal to the date of death,
but this is a phenomenon that was observed for any
AE checked.
3.4.1 Why does this matter?
This corroborates the hypothesis that there is a lag-
phase between reporting and recording of data. The
duration between reporting following onset of an
AE reaction and recording into the VAERS publicly
available data varies from a few days to many
months. Figure 8 shows the difference in data with
respect to the data as per weekly update and to the
updated data as of August 6th, 2021, for all SAEs.
The black shaded area represents data that is in
excess with regards to the data originally presented
to the public. The data under the blue line is the
15
Onset Date: The date of the onset of adverse event symptoms associated with the vaccination as recorded in the
specified field of the form.
16
Today’s date: The date the form was completed.
most recently update data and the data under the red
line is the weekly updated data. The most alarming
observation from this figure, however, is the
amount of data that was present early on that simply
was not publicly available at the time that they were
generated. For example, the ∆ cumulative AEs
between the individual updated data for week 10 is
19,536. The ∆ time in weeks is 7.6. This means that
almost 20,000 SAEs that should be observable in
the publicly available VAERS Domestic dataset
were not present at the time they occurred and were
originally reported. This means that only 7,065
(red)/26601 (blue) = ~20% of the actual SAEs as of
that date (week 1) were entered into the database.
Only after a lag time of almost 2 months did this
data become visible. If week 5 is examined, this lag-
time becomes 10 weeks (Figure 8 - right). It is only
recently that these data were made visible and this
is most likely due to a huge backlog being tended
to. The fact that the data sets have converged is due
to the backlog being sufficiently dealt with. This
phenomenon was found to exist to varying degrees
in all AEs checked. Figure 9 shows 3 representative
plots for Chills, Death and Breakthrough COVID-
19 AEs. It is fortunate (in a way) that the death data
does not seem to have been a victim of the lag like
some others. This phenomenon was also not
dependent on an AE being mild or severe but the
degree to which the phenomenon occurred in each
AE is yet to be ascertained. This can be checked.
Another way to assess temporal differences in
data entry is to calculate the number of days
between the onset date (ONSET_DATE)
15
and the
date that the AE was input into the VAERS
database (TODAY’S_DATE)
16
using only the most
recent updated file. For example, the difference
between the completed form entry date and the
onset of the AE date should be the same for any two
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
111
Figure 9: Shaded plots showing the Chills, Death and Breakthrough COVID AE data as they were
input per respective update (grey shaded region) compared with these data as they are reported in each
individual updated file (black)
randomly selected AEs. If there was a difference
between the percentages of reports made for any
two AEs, based on the difference between entry
date and onset of AE date, then this would require
explanation, especially if the difference was
statistically significant. The most frequently
reported AE in the VAERS system in the context of
COVID-19 products is “Chills”. I chose this AE as
a positive control against deaths in the context of
whether or not these two AE types were being
added to the publicly available VAERS database in
the same way, temporally.
Figure 10 shows the percentages of reported
Deaths and Chills as a starting point for the
comparison. The T-test confirms a statistically
significant difference between the respective means
of the Death and Chills AEs with regards to differences
Figure 10: Time series plot showing percentages
of Chills (green/yellow) and Death (green/red) of
the total VAERS dataset (as of update July 30th,
2021) against the number of days calculated in
between the entry date of the report into the
database and the onset date of AE for up to 15
days’ difference
Figure 8: Shaded plots showing the SAE data as they were input per respective update (grey shaded
region) compared with these data as they are reported in each individual updated file (black)
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
112
in reporting times following onset of AE with a p-
value = 0.005. The figures show areas under the
curves generated to demonstrate how many more
entries were made in the case of Chills than for
Death within the first 5 days following onset of AE.
3.4.2 Lag time dependency on AE type?
Figure 11 shows the percentages of reported
Deaths, Bell’s palsy, Heavy menstrual bleeding,
Myocarditis, Injection site pruritis, Chills,
Headache, and Fatigue data against the differences
in days between their onset dates and the entry dates
into the Domestic front-end VAERS system that is
Figure 11: Time series plot showing percentages
of reported Headache (H), Chills (Ch), Injection
site pruritis (ISP), Fatigue (F), Dizziness (D)
(blue), Bell’s palsy (BP), Death (D), Heavy
menstrual bleeding (HMB), Foetal death (FD),
COVID-19 (C19) (red) of the total VAERS dataset
(as of update July 30th, 2021) against the number
of days calculated in between the entry date of the
report and the onset date of AE
available for download. These 10 were selected
since 5 are classified as severe and 5 are classified
as mild.
There is a clear difference in the percentages of
reports made between the mild AEs: Headache (H),
Chills (Ch), Injection site pruritis (ISP), Fatigue (F)
and Dizziness (D) and severe AEs: Bell’s palsy
(BP), Death (D), Heavy menstrual bleeding (HMB),
Foetal death (FD), COVID-19 (C19). In the case of
the mild AEs listed, the area under the curves
(AUCs) are greater than the AUCs in the first few
days following the onset of the AE. In the cases of
the more severe AEs, <10% of reports were entered
within the first few days. It is yet unclear whether
or not this is a coincidence.
3.5 Are SAEs being downgraded to MAEs each
week?
The rate of SAE occurrence according to VAERS
data is 19% (nSAE/N reports to VAERS (%)). If we
use only Pfizer data, this rate increases to 21%. If
we normalize to dose number, we get 0.02% rate of
SAE (nSAE/N doses) so this translates to ~1/5000
individuals succumbing to a SAE. There is
variation between the criteria that the CDC uses to
determine SAEs in VAERS and the medical
definition of SAEs [4,5,6,7]. This raises the
question of whether specific SAE reports in
VAERS are downgraded over time to MAEs. The
short answer is no. To determine whether or not
SAEs were being downgraded to mild AEs, I
semi-joined the datasets for a selected update date
Table 1: Calculated SAE and MAE differences between reference file and original file for 10 sample
update files downloaded from VAERS
Reference Update (RU)
∆ (date-RU) 03_05_21 03_12_21 03_19_21 03_26_21 04_02_21 04_09_21 04_16_21 04_23_21 04_30_21 05_07_21 05_14_21 21_05_21
∆total AE count 86 118 105 124 94 94 93 120 162 149 77 0
∆SAE count 14 38 35 49 23 28 28 53 107 98 49 0
∆MAE count 72 80 70 75 71 66 65 67 55 51 28 0
∆% SAE 0 0 0 0 0 0 0 0 0 0 0 0
∆% MAE 0 0 0 0 0 0 0 0 0 0 0 0
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
113
(03/05/21) with 10 sequential updates to maintain
the same smaller cohort within the data frames. This
allowed the comparison of the original SAE and
MAE counts to the original counts for the
individual dataframes to check if the counts were
changing as updates were being added. None of the
SAE counts were different when compared to semi-
joined dataframes which means that SAEs are not
being downgraded to mild AEs as the updates come
in (Table 1). The discrepancies in deltas seen in
adverse events (and thus both SAEs and MAEs) are
most likely due to variations in data reporting and
recording that are known.
4
Discussion
Functioning pharmacovigilance in VAERS was
examined in this study. It appears from this short
appraisal that although VAERS could be a
functioning pharmacovigilance system, it is not
being used as such. The only reference to legitimate
deletion of data from the VAERS system was in the
VAERS/WONDER ‘Reporting Issues’ section,
which claims that ‘Duplicate event reports and/or
reports determined to be false are removed from
VAERS’. Despite this ‘disclaimer’, there is no way
to check or validate ‘falseness’ of data that may
have been removed. This means that, in the case of
deleted deaths, which represent 3% of all death
data, their removal needs to be explained. These
deaths were reported to VAERS and recorded by
hired CDC contractors. They represent people who
died in temporal proximity to having been given an
as-yet non-FDA-approved, experimental transfective
biological product by intramuscular injection. They
cannot simply be deleted. Something worth noting
was the commonality in deleted entries where a
causality relationship between the injections and
the AE was not only implied but also suggested by
the sender, which is typically the physician or
emergency-room physician who attended to the
individual’s case. Refer to Supplementary Table 1
for deleted death entries in the VAERS Wayback
machine.
Trained contractor staff are required to enter
each VAERS report into the database, and if it
should be deemed necessary to delete a VAERS ID
from this database once entered, then it must be
documented with a valid reason for the deletion. In
addition, when a VAERS ID number is changed to
a new number, this should also be documented by
contractor staff. It has been suggested that vaccine-
induced deaths have been classified as COVID-19
deaths. If this is the case, then deaths are being
skewed away from the elusive vaccine-induced
death count toward the COVID-19 death count
[33,34]. It is unscientific to deny any possibility that
the injections are the possible cause of the injuries,
particularly in some cases where the clear temporal
proximity makes this possibility a high probability
[8,35]. If this denial was implemented into a system
of denial, it would most likely manifest in this way.
VAERS was designed to reveal potential risk
signals from data, but if these signals are not
detectable as they are received, then they are not
useful as timely warnings. There is evidence that
the VAERS data are being entered into the publicly
available dataset much later than one would expect,
considering that this is a passive system. It is
conceivable that death AEs have extended
processing times for the issuance of death
certificates, but there would be no reason for other
AEs, severe or mild, to have delays with regards to
data entry, especially not delays greater than 4
weeks. Public health policy decisions on expanding
the vaccination program might have been made
differently if the true rates of reported SAEs and
deaths had been known in real time. Similarly, if
individuals knew of SAEs and deaths occurring so
early on in the rollout, and also that the percentage
of SAEs is atypically high, then perhaps they would
have exercised their rights to informed consent,
declined these injections or simply waited for safety
data to come in. This is precisely what the VAERS
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
114
system is designed for in its pharmacovigilance
task: to warn policy makers and individuals of
potential risks not detected during clinical trials. If
there is a large backlog of data, then more trained
staff need to be hired to expedite data entry to
ensure that the VAERS system is able to deliver
safety signals as they are reported. In the case where
late entry of data occurs due to another reason, then
this needs to be acknowledged, investigated and
remedied. The evidence provided herein lends to
the hypothesis that data is being entered according
to AE severity. This alone requires investigation.
As a point of concern with regards to CDC safety
signal metrics, as defined in section 2.3.1 in the
SOP, the proportional reporting ratio (PRR) is used
to define safety signals originating from VAERS.
The PRR is a metric that compares the ratio of
specific AEs to total AEs for vaccine products. It is
defined as:
where a = specific AE for specific vaccine; b = all
other AEs for specific vaccine; c = specific AE for
all other vaccines; d = all other AEs for all other
vaccines [36,37]. However, this technique is
inherently flawed in that the PRR does not change
when the specific vaccine-related AE event counts
are very large or very small [34,36,37,38].
Therefore, the scaling factor that arises due to the
excess of specific AEs is normalized to the total
number of AEs, and this ratio is then again
normalized to the total for all other vaccines. This
is a problem in the context of the COVID-19
injectable products since both the specific AEs and
the total number of AEs are atypically high. This
means that no matter how many times higher the
death rate, for example, the PRR will be the same
as it would be for a product that was not killing
people at all. The PRR, therefore, on its own, cannot
be used as reliable a safety signal detection metric
– it does not work.
To be clear, the absolute number of AEs reported
in the context of the COVID-19 products is
approximately 11x higher than for all the reported
AEs for 2020 combined. The absolute number of
deaths reported is approximately 42x higher than
for all deaths reported for 2020. However, the PRR
does not emit a safety signal even though the
number of deaths is 266 times higher in the context
of the COVID-19 products when compared to
INFLUENZA products [32]. In spite of peer-
reviewed studies noting significant association of
COVID-19 injectable products with Bell’s palsy,
thrombocytopenia and myocarditis [39,40,41,42],
the CDC maintains the position that no specific
safety concerns have been identified with regards to
SAEs [8,31,43,44,45]. In a recent CDC report titled
‘Local Reactions, Systemic Reactions, Adverse
Events, and Serious Adverse Events: Pfizer-
BioNTech COVID-19 Vaccine’ [44], only the
severity of the most frequently reported AEs in the
VAERS database are reported in tabular form and
not the SAEs themselves. They report that
occurrence of SAEs involving system organ classes
and specific preferred terms were balanced between
vaccine and placebo groups and presented at a mere
0.5%, and although SAEs (grade ≥3, defined as
interfering with daily activity) occurred more
commonly in vaccine recipients than in placebo
recipients, their claim is that no specific safety
concerns were identified with regards to SAEs,
which is false [43,44,45].
One more discussion point that is worth its own
publication but will be added as a point of interest
in this study is the Under-Reporting Factor (URF)
of AEs. Under-reporting is a problem in
pharmacovigilance systems, VAERS included.
VAERS is a passive reporting system, and it has
been suggested as part of a Harvard study that a
mere 1% of AEs are reported to VAERS [46].
However, this is not necessarily the case, nor is it
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
115
universally applicable for all products; certainly not
for distinct AEs. For example, under-reporting of
mild AEs such as rashes or low-grade fever would
most likely be far greater than for SAEs, such as
death. To calculate the URF, the expected number
of SAEs (ESAE) is divided by the observed number
of SAEs (OSAE). The ESAE is calculated by
multiplying the total number of doses administered
in the U.S. (assuming a single dose can result in an
AE) by the number of SAEs recorded in COVID-
19 product safety trials. According to the FDA
Safety Overview of the Pfizer/BioNTech COVID-
19 product (Study C4591001 – refer to section 5.2.6
page 33) [47,48]. 0.7% of Pfizer/BioNTech
COVID-19 product recipients suffered SAEs. As of
August 10th, 2021, 197,399,471 million Pfizer/
BioNTech COVID-19 product doses had been
administered in the U.S. [49,50] and therefore the
number of expected SAE occurrences in the U.S.
volunteer recipients of the Pfizer/BioNTech
products should be ~1.4 million SAEs, if we use
this reported rate. Thus, the ratio of ESAE to OSAE is
31 to 1, suggesting a URF of 31
(NSAE_Pfizer_trial/NSAE_Pfizer_VAERS = ~1.4M/43,948).
Using this URF for all VAERS-classified SAEs,
estimates to date are as follows: 205,809 dead,
818,462 hospitalizations, 1,830,891 ER visits,
230,113 life-threatening events, 212,691 disabled
and 7,998 birth defects to date [38]. Since the URF
for MAEs is very likely larger than for SAEs, it is
satisfactory to assume that 31 is a humble estimate
URF for all AEs (refer to Supplementary Table 2).
Relative reporting rates are also shown in
Supplementary Table 2 to demonstrate that that AE
reports associated with COVID-19 products are
much higher than for previous years. For all
symptoms listed in red, we limited the search to 20–
60-year-olds since these people are less noisy with
respect to symptoms and younger people aren’t yet
vaccinated. All fields color-coded yellow contain
observed/expected incidence rates >100, and these
only occur in the non-control AEs, such as reported
AEs that are presumably unrelated to the vaccines,
like ‘Lyme disease’, seen in blue and green in
Supplementary Table 2.
5
Conclusion
It cannot be stressed enough when referring to
VAERS data collected in the context of the
COVID-19 injectable products that effective
antiviral responses against the nCoV-2019 virus in
the form of both cellular and humoral immune
responses have been reported in peer-reviewed
studies [51–56]. Because of the low Infection
Fatality Rate, indicating effective and robust
immune responses, it remains unclear why multiple
experimental mRNA vaccines have been fast-
tracked through conventional testing protocols and
are also being fast-tracked through production and
administration into the public. With repurposed
drugs like hydroxychloroquine and Ivermectin
showing extremely positive results in patients [57–
68], it is also unclear why these drugs are not being
more extensively promoted as effective tools in the
fight against this virus. What is clear is that the
injectable products are proving unsafe for many
individuals and inefficacious in others (see Israeli
data in Supplementary Material). As part of the
WHO’s own minimum requirements for a
functioning pharmacovigilance system, sub-
standard products need to be removed from
circulation to ensure patient safety. Since VAERS
is capable as a functioning pharmacovigilance
system as it reveals safety issues with the COVID-
19 biologicals, it should be used as such, but it is
not.
Despite the low frequency of missing VAERS
IDs, data have been deleted from the VAERS
database, and this requires explanation, not only
ethically but also because it lends to the possibility
of inexact measurements of death counts and
therefore can potentially lead to missed signals.
Statistical power is primarily influenced by sample
size (also effect size and significance level), and the
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
116
bigger the sample size, the higher the statistical
power. The deleted data from the total VAERS ID
count are individuals enrolled in post-market
surveillance human-subject studies: the where-
abouts of their VAERS reports of death need to be
accounted for. There is absolutely no reason for
these data to be missing, from what can be
ascertained. If the data were false, as was suggested
as the only reason to delete an entry, then there
needs to be a record of this edited data made
available with the publicly available VAERS data.
Data are being retroactively added to the
VAERS database far later than would be expected
for the system to be considered a timely,
functioning pharmacovigilance system. This could
be explained by manual curation of a large backlog
of data. However, if AEs are being entered
differentially, with respect to time, based on
severity, then we all must ask the difficult question:
“Why?” Again, VAERS was designed to reveal
potential risk signals from data, but if these signals
are not detectable as they are received, then they are
not useful as warnings and pharmacovigilance
becomes moot. The duration between reporting
following onset of an adverse event reaction and
recording into the VAERS publicly available data
varies from a few days to many months. If earlier
information was available to public health policy-
makers and to the public, including the off-the-
charts prevalence of SAEs (19%) and deaths, then
perhaps the decision to volunteer to have these
products injected would have been more
prevalently declined or simply put on hold until
more safety data had accumulated. This, again, is
part of pharmacovigilance that has failed with
regards to assessment of risk/benefit management.
According to this analysis, VAERS IDs are not
being downgraded from SAEs to mild AEs. In fact,
the percentage of SAEs continue to increase from
month to month. Even without considering the
URF, the ratio of fully vaccinated individuals
succumbing to an adverse event is high. With
approximately 1 in every 400 individuals
experiencing an adverse event (~1 in every 25,000
for death) in the context of the COVID-19 fully
vaccinated population in the United States, it is
therefore unclear why these injections are
continuing to be used in the human population,
especially since no long-term effects are known and
no long-term data exists, to date. It was important
to contextualize death counts since a dis-
proportionate number of all the missing data AEs
are deaths.
It may appear that the number of missing
VAERS IDs is nothing to be concerned about from
an analytical point of view, but I remind the reader
that these are not just data: they are people. This
report addressed three issues that respond to the
question of VAERS pharmacovigilance by
analyzing VAERS data in relation to: 1. deleted
reports, 2. delayed entry of reports, and 3. recoding
of MedDRA terms from severe to mild.
6
References
1. World Health Organization. 2010. Minimum
requirements for a functional pharmaco-
vigilance system.
http://www.who.int/medicines/areas/quality_sa
fety/safety_efficacy/PV_Minimum_Requireme
nts_presentation.ppt
2. World Health Organization. 2021. Pharmaco-
vigilance.
https://www.who.int/teams/regulation-
prequalification/regulation-and-
safety/pharmacovigilance
3. U.S. Department of Health and Human
Services, Food and Drug Administration,
Center for Drug Evaluation and Research
(CDER), Center for Biologics Evaluation and
Research (CBER). March 2005. Guidance for
industry: Good pharmacovigilance practices
and pharmacoepidemiologic assessment.
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
117
https://www.fda.gov/regulatory-
information/search-fda-guidance-
documents/good-pharmacovigilance-practices-
and-pharmacoepidemiologic-assessment
4. U.S. Department of Health and Human
Services. 2020. VAERS Data Use Guide.
https://vaers.hhs.gov/docs/VAERSDataUseGui
de_November2020.pdf
5. Varricchio F, Iskander J, Destefano F, Ball R,
Pless R, Braun MM, Chen RT. Understanding
vaccine safety information from the Vaccine
Adverse Event Reporting System. Pediatr
Infect Dis J. 2004 Apr;23(4):287–94.
doi: 10.1097/00006454-200404000-00002.
PMID: 15071280.
6. Iskander JK, Miller ER, Chen RT. 2004. The
role of the Vaccine Adverse Event Reporting
system (VAERS) in monitoring vaccine safety.
Pediatr Ann. 33(9):599–606.
doi: 10.3928/0090-4481-20040901-11.
PMID: 15462575.
7. VAERS Team: Immunization Safety Office,
Division of Healthcare Quality Promotion
National Center for Emerging and Zoonotic
Infectious Diseases and Centers for Disease
Control and Prevention. 2021.Vaccine Adverse
Event Reporting System (VAERS), Standard
Operating Procedures for COVID-19 (as of 29
January 2021).
https://www.cdc.gov/vaccinesafety/pdf/VAER
S-v2-SOP.pdf
8. Rose, J. 2021. A report on the US Vaccine
Adverse Events Reporting System (VAERS) of
the COVID-19 messenger ribonucleic acid
(mRNA) biologicals. Sci Publ Health Pol &
Law 2:59–80.
9. Rosenthal S, Chen R. 1995. The reporting
sensitivities of two passive surveillance systems
for vaccine adverse events. Am J Public Health,
1995 Dec;85(12):1706-9.
doi: 10.2105/ajph.85.12.1706.
https://www.ncbi.nlm.nih.gov/pmc/articles/PM
C1615747/pdf/amjph00450-0108.pdf
10. Centers for Disease Control and Prevention.
2021. Interim Public Health Recommendations
for Fully Vaccinated People.
https://www.cdc.gov/coronavirus/2019-
ncov/vaccines/fully-vaccinated-guidance.html
11. U.K. Government. Reg. 174 Information for UK
Healthcare Professionals.
https://assets.publishing.service.gov.uk/govern
ment/uploads/system/uploads/attachment_data/
file/1016211/Temporary_Authorisation_HCP_
Information_BNT162_-_09-09-2021.pdf
12. Food and Drug Administration. 2021. Fact
Sheet for Vaccination Providers Administering
Vaccine.
https://www.fda.gov/media/144413/download
13. World Health Organization. 2010. Questions
and answers relating to finding of porcine
circoviruses in rotavirus vaccine.
https://www.who.int/immunization_standards/
vaccine_quality/PCV1_Q_and_As_rotavirus_v
accines_3Jun10.pdf
14. Food and Drug Administration. 2010. Update
on Recommendations for the Use of Rotavirus
Vaccines.
http://www.fda.gov/BiologicsBloodVaccines/V
accines/ApprovedProducts/ucm205540.htm;
https://www.pediatrics.org.il/guids/FDA_Rota
virus_Vaccines.pdf
15. Centers for Disease Control and Prevention.
2020. Historical Vaccine Safety Concerns.
https://www.cdc.gov/vaccinesafety/concerns/c
oncerns-history.html
16. Duffy J, Weintraub E, Vellozzi C, DeStefano F;
Vaccine Safety Datalink. Narcolepsy and
influenza A(H1N1) pandemic 2009 vaccination
in the United States. Neurology. 2014 Nov;
11;83(20):1823–30.
doi: 10.1212/WNL.0000000000000987.
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
118
Epub 2014 Oct 15. PMID: 25320099; PMCID:
PMC6563919.
17. Weibel D, Sturkenboom M, et al. 2018.
Narcolepsy and adjuvanted Pandemic Influenza
A (H1N1) 2009 Vaccines – Multi-country
Assessment. Vaccine. 1;26(41):6202–6211.
18. Velentgas P, Amato AA, Bohn R, et al. 2012.
Risk of Guillain-Barré syndrome after
meningococcal conjugate vaccination.
Pharmacoepidemiol Drug Saf. 21(12):1350–8.
19. Centers of Disease Control and Prevention.
1999. Withdrawal of rotavirus vaccine
recommendation. MMWR 48(43);1007.
https://www.cdc.gov/mmwr/preview/mmwrht
ml/mm4843a5.htm
20. Parashar UD, Alexander JP, Glass RI, CDC,
ACIP. 2006. Prevention of rotavirus
gastroenteritis among infants and children.
Recommendations of the Advisory Committee
on Immunization Practices (ACIP). MMWR
Recomm Rep 2006 Aug 11;55(RR-12):1–13.
21. Cortese M, Parashar UD. 2009. Prevention of
rotavirus gastroenteritis among infants and
children: Recommendations of the Advisory
Committee on Immunization Practices (ACIP).
MMWR February 6, 2009 / 58(RR02);1–25.
22. Centers for Disease Control and Prevention.
1999. Withdrawal of rotavirus vaccine
recommendation. MMWR. November 05, 1999
/ 48(43);1007.
23. Centers of Disease Control and Prevention.
1999. Intussusception among recipients of
rotavirus vaccine — United States, 1998–1999.
MMWR. July 16, 1999 / 48(27);577–581.
24. Patel MM, Haber P, Baggs J, Zuber P, Bines JE,
Parashar UD. 2009. Intussusception and
rotavirus vaccination: A review of the available
evidence. Expert Rev Vaccines 8(11), 1555–
1564.
25. Tate JE, Simonsen L, Viboud C, Steiner C, Patel
MM, Curns AT, Parashar UD. 2008. Trends in
Intussusception Hospitalizations among US
Infants, 1993–2004: Implications for
monitoring the safety of the new rotavirus
vaccination program. Pediatrics 121(5), e1125–
1132.
26. Murphy TV, Gargiullo PM, Massoudi MS,
Nelson DB, Jumaan AO, Okoro CA, Zanardi
LR, Setia S, Fair E, LeBaron CW, Wharton M,
Livengood JR; Rotavirus Intussusception
Investigation Team. 2001. Intussusception
among infants given an oral rotavirus vaccine.
New England Journal of Medicine 344(8):564–
572.
27. Zanardi LR, Haber P, Mootrey GT, Niu MT,
Wharton M. 2001. Intussusception among
recipients of rotavirus vaccine: reports to the
Vaccine Adverse Event Reporting System.
Pediatrics 107(6):E97.
28. Kramarz P, France EK, Destefano F, Black SB,
Shinefield H, Ward JI, Chang EJ, Chen RT,
Shatin D, Hill J, Lieu T, Ogren JM. 2001.
Population-based study of rotavirus vaccination
and intussusception. Pediatric Infectious
Disease Journal 20(4):410–416.
29. Rennels MB. 2000. The rotavirus vaccine story:
a clinical investigator’s view. Pediatrics
106:123–5.
30. Institute of Medicine (US) Immunization Safety
Review Committee. 2002. Immunization Safety
Review: Hepatitis B vaccine and demyelinating
neurological disorders. Stratton K, Almario D,
McCormick MC, editors. Washington (DC):
National Academies Press (US). PMID:
25057609.
31. Advisory Committee on Immunization
Practices (ACIP), Shimabukuro T. CDC
COVID-19 Vaccine Task Force. June 23, 2021.
COVID-19 Vaccine safety updates.
https://www.cdc.gov/vaccines/acip/meetings/d
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
119
ownloads/slides-2021-06/03-COVID-
Shimabukuro-508.pdf
32. Guetzkow J. 2021. Adverse Events Reported
Following COVID-19 Vaccinations.
https://tinyurl.com/CovidvFluReport
33. Crawford M. 2021. Estimating vaccine-induced
mortality, Part II: Isolating the variable. The
Chloroquine Wars Part LIII.
https://roundingtheearth.substack.com/p/estima
ting-vaccine-induced-mortality-e07
34. Crawford M. 2021 Defining away vaccine
safety signals: The Chloroquine Wars Part
XLVIII.
https://roundingtheearth.substack.com/p/defini
ng-away-vaccine-safety-signals
35. IPAK Report 2021–1. 2021. Post-vaccination
death causality likely given temporal
distribution of deaths following COVID-19
vaccinations. Interim results.
http://ipaknowledge.org/resources/VAERS%20
deaths%20to%203%2010%202021%20update
%203.pptx
36. Evans SJ, Waller PC, Davis S. 2001. Use of
proportional reporting ratios (PRRs) for signal
generation from spontaneous adverse drug
reaction reports. Pharmacoepidemiol Drug Saf.
Oct–Nov;10(6):483–6. doi: 10.1002/pds.677.
PMID: 11828828.
37. Du J, Cai Y, Chen Y, He Y, Tao C. 2017.
Analysis of individual differences in vaccine
pharmaco-vigilance using VAERS data and
MedDRA system organ classes: A use case
study with trivalent influenza vaccine. Biomed
Inform Insights. Apr 11, 2017; 9:1–8.
doi: 10.1177/1178222617700627.
PMID: 28469434; PMCID: PMC5391193.
38. Malone R, Kirsch S, Bridle B, Seneff S,
Crawford M, Rose J. VACCINE SAFETY
FAQ.
39. Repajic M, Lai XL, Xu P, Liu A. 2021. Bell's
Palsy after second dose of Pfizer COVID-19
vaccination in a patient with history of recurrent
Bell's palsy. Brain Behav Immun Health.
13:100217. doi:10.1016/j.bbih.2021.100217.
40. Novak N, Tordesillas L, Cabanillas B. 2021.
Adverse rare events to vaccines for COVID-19:
From hypersensitivity reactions to thrombosis
and thrombocytopenia. Int Rev Immunol. Jul 12,
2021:1–10. doi: 10.1080/08830185.2021.1939696.
Epub ahead of print. PMID: 34251972; PMCID:
PMC8290371.
41. Welsh KJ, Baumblatt J, Chege W, Goud R, Nair
N. 2021. Thrombocytopenia including immune
thrombocytopenia after receipt of mRNA
COVID-19 vaccines reported to the Vaccine
Adverse Event Reporting System (VAERS).
Vaccine. 39(25):3329–3332.
doi: 10.1016/j.vaccine.2021.04.054. Epub 2021
Apr 30. PMID: 34006408; PMCID:
PMC8086806.
42. Minocha PK, Better D, Singh RK, Hoque T.
2021. Recurrence of acute myocarditis
temporally associated with receipt of the mRNA
coronavirus disease 2019 (COVID-19) vaccine
in a male adolescent [published online ahead of
print, 2021 Jun 22]. J Pediatr. S0022–
3476(21)00617-X. doi: 10.1016/j.jpeds.2021.
06.035.
43. Centers for Disease Control and Prevention.
2021, May 4. Local reactions, systemic
reactions, adverse events, and serious adverse
events: Pfizer-BioNTech COVID-19 vaccine.
https://www.cdc.gov/vaccines/covid-19/info-
by-product/pfizer/reactogenicity.html
44. Centers of Disease Control and Prevention.
2021. The Advisory Committee on Immuniz-
ation Practices’ interim recommendation for use
of Pfizer-BioNTech COVID-19 vaccine in
adolescents Aged 12–15 Years — United
States, May 2021.
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
120
https://www.cdc.gov/mmwr/volumes/70/wr/m
m7020e1.htm
45. Klein N, Kaiser Permanente Vaccine Study
Center, Kaiser Permanente Northern California,
Marshfield Clinic Research Institute, Vaccine
Safety Datalink – Immunization Safety Office,
CDC. 2021. Rapid Cycle Analysis (RCA) to
monitor the safety of COVID-19 Vaccines in
near real-time within the Vaccine Safety
Datalink: Guillain-Barré Syndrome (GBS).
https://www.cdc.gov/vaccines/acip/meetings/d
ownloads/slides-2021-07/03-COVID-Klein-
508.pdf
46. Lazarus, Ross et al. Grant Final Report. Grant
ID: R18 HS 017045. Electronic Support for
Public Health–Vaccine Adverse Event
Reporting System (ESP:VAERS). Submitted to
The Agency for Healthcare Research and
Quality (AHRQ).
47. Vaccines and Related Biological Products
Advisory Committee Meeting. December 10,
2020. FDA Briefing Document Pfizer-
BioNTech COVID-19 Vaccine.
https://www.fda.gov/media/144245/download
48. Crawford M. 2021. How Underreported Are
Post-Vaccination Serious Injuries and Deaths in
VAERS? The Chloroquine Wars Part LI.
https://roundingtheearth.substack.com/p/how-
underreported-are-post-vaccination
49. Mathieu E, Ritchie H., Ortiz-Ospina E, et al.
2021. A global database of COVID-19
vaccinations. Nature Human Behavior 5:947–
953.
50. Haseman J. 2021. Tracking COVID-19 vaccine
distribution by state: How many people have
been vaccinated in the US? Source: CDC data.
https://www.usatoday.com/in-
depth/graphics/2021/01/14/covid-vaccine-
distribution-by-state-how-many-covid-
vaccines-have-been-given-in-us-how-many-
people/6599531002/
51. Toor SM, Saleh R, Sasidharan Nair V, Taha RZ,
Elkord E. 2021. T-cell responses and therapies
against SARS-CoV-2 infection. Immunology.
162(1):30–43. doi: 10.1111/imm.13262. Epub
2020 Oct 27. PMID: 32935333; PMCID:
PMC7730020.
52. Robbiani DF, et al. 2020. Convergent antibody
responses to SARS-CoV-2 in convalescent
individuals. Nature 584(7821):437–442. doi:
10.1038/s41586-020-2456-9. Epub 2020 Jun
18. PMID: 32555388; PMCID: PMC7442695.
53. Sun B, et al. 2020. Kinetics of SARS-CoV-2
specific IgM and IgG responses in COVID-19
patients. Emerg Microbes Infect. 9(1):940–948.
doi: 10.1080/22221751.2020.1762515. PMID:
32357808; PMCID: PMC7273175.
54. Le Bert N, et al. 2020. SARS-CoV-2-specific T
cell immunity in cases of COVID-19 and SARS,
and uninfected controls. Nature 584(7821):457
–462. doi: 10.1038/s41586-020-2550-z. Epub
2020 Jul 15. PMID: 32668444.
55. Mateus J, et al. 2020. Selective and cross-
reactive SARS-CoV-2 T cell epitopes in
unexposed humans. Science 370(6512):89–94.
doi: 10.1126/science.abd3871. Epub 2020 Aug
4. PMID: 32753554; PMCID: PMC7574914.
56. Lipsitch M, Grad YH, Sette A, Crotty S. 2020.
Cross-reactive memory T cells and herd
immunity to SARS-CoV-2. Nat Rev Immunol.
20(11):709–713. doi: 10.1038/s41577-020-
00460-4. Epub 2020 Oct 6. PMID: 33024281;
PMCID: PMC7537578.
57. Colson P, Rolain JM, Lagier JC, Brouqui P,
Raoult D. 2020. Chloroquine and
hydroxychloroquine as available weapons to
fight COVID-19. Int J Antimicrob Agents.
55(4):105932.
doi: 10.1016/j.ijantimicag.2020.105932. Epub
2020 Mar 4. PMID: 32145363; PMCID:
PMC7135139.
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
121
58. Meo SA, Klonoff DC, Akram J. 2020. Efficacy
of chloroquine and hydroxychloroquine in the
treatment of COVID-19. Eur Rev Med
Pharmacol Sci. 24(8):4539–4547. doi:
10.26355/eurrev_202004_21038. PMID:
32373993.
59. Ibáñez S, Martínez O, Valenzuela F, Silva F,
Valenzuela O. 2020. Hydroxychloroquine and
chloroquine in COVID-19: Should they be used
as standard therapy? Clin Rheumatol.
39(8):2461-2465. doi: 10.1007/s10067-020-
05202-4. Epub 2020 Jun 3. PMID: 32495226;
PMCID: PMC7267470.
60. N, Esposito S. 2020. Chloroquine or
hydroxychloroquine for prophylaxis of
COVID-19. Lancet Infect Dis. 20(10):1118.
doi: 10.1016/S1473-3099(20)30296-6. Epub
2020 Apr 17. PMID: 32311322; PMCID:
PMC7164862.
61. Ferner RE, Aronson JK. 2020. Chloroquine and
hydroxychloroquine in Covid-19. BMJ.
369:m1432. doi: 10.1136/bmj.m1432. PMID:
32269046.
62. Hernandez AV, Roman YM, Pasupuleti V,
Barboza JJ, White CM. 2020. Hydroxy-
chloroquine or Chloroquine for treatment or
prophylaxis of COVID-19: A living systematic
review. Ann Intern Med. 173(4):287–296. doi:
10.7326/M20-2496. Epub 2020 May 27. PMID:
32459529.;
63. Shah S, Das S, Jain A, Misra DP, Negi VS.
2020. A systematic review of the prophylactic
role of chloroquine and hydroxychloroquine in
coronavirus disease-19 (COVID-19). Int J
Rheum Dis. 23(5):613–619. doi: 10.1111/1756-
185X.13842. Epub 2020 Apr 27. PMID:
32281213; PMCID: PMC7262257.
64. Rizzo E. 2020. Ivermectin, antiviral properties
and COVID-19: a possible new mechanism of
action. Naunyn Schmiedebergs Arch Pharmacol.
393(7):1153–1156. doi: 10.1007/s00210-020-
01902-5. Epub 2020 May 27. PMID: 32462282;
PMCID: PMC7251046.
65. Heidary F, Gharebaghi R. 2020. Ivermectin: a
systematic review from antiviral effects to
COVID-19 complementary regimen. J Antibiot
(Tokyo). 73(9):593–602. doi: 10.1038/s41429-
020-0336-z. Epub 2020 Jun 12. PMID:
32533071; PMCID: PMC7290143.
66. Sharun K, Dhama K, Patel SK, Pathak M,
Tiwari R, Singh BR, Sah R, Bonilla-Aldana
DK, Rodriguez-Morales AJ, Leblebicioglu H.
2020. Ivermectin, a new candidate therapeutic
against SARS-CoV-2/COVID-19. Ann Clin
Microbiol Antimicrob. 19(1):23. doi:
10.1186/s12941-020-00368-w. PMID:
32473642; PMCID: PMC7261036.
67. Shih RD, Johnson HM, Maki DG, Hennekens
CH. 2020. Hydroxychloroquine for corona-
virus: The urgent need for a moratorium on
prescriptions. Am J Med. 133(9):1007–1008.
doi: 10.1016/j.amjmed.2020.05.005. Epub 2020
Jun 2. PMID: 32502485; PMCID:
PMC7265864.
68. Lam S, Lombardi A, Ouanounou A. 2020.
COVID-19: A review of the proposed
pharmacological treatments. Eur J Pharmacol.
886:173451. doi: 10.1016/j.ejphar.2020.173451.
Epub 2020 Aug 6. PMID: 32768505; PMCID:
PMC7406477.
69. Dagan N, et al. 2021. BNT162b2 mRNA Covid-
19 Vaccine in a Nationwide Mass Vaccination
Setting. New England Journal of Medicine.
384:1412–1423.
DOI: 10.1056/NEJMoa2101765.
70. Zimmermann P, Curtis N. 2019. Factors that
influence the immune response to vaccination.
Clin Microbiol Rev. 32(2):e00084–18. doi:
10.1128/CMR.00084-18. PMID: 30867162;
PMCID: PMC6431125.
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
122
71. Nath TR, Malaviya AN, Kumar R, Balakrishnan
K, Singh BP. 1997. A study of the efficacy of
typhoid vaccine in inducing humoral and cell-
mediated immune responses in human
volunteers. Clin Exp Immunol. 30(1):38–43.
72. Orenstein WA, Bernier RH, Dondero TJ,
Hinman AR, Marks JS, Bart KJ, Sirotkin B.
1985. Field evaluation of vaccine efficacy. Bull
World Health Organ. 63(6):1055–68. PMID:
3879673; PMCID: PMC2536484.
73. Furman D, Davis MM. 2015. New approaches
to understanding the immune response to
vaccination and infection. Vaccine 33(40):
5271–81. doi: 10.1016/j.vaccine.2015. 06.117.
Epub 2015 Jul 29. PMID: 26232539; PMCID:
PMC4581990.
74. Demeure CE, Derbise A, Guillas C, Gerke C,
Cauchemez S, Carniel E, Pizarro-Cerdá J. 2019.
Humoral and cellular immune correlates of
protection against bubonic plague by a live
Yersinia pseudotuberculosis vaccine. Vaccine.
37(1):123–129. doi: 10.1016/j.vaccine.2018.11.
022. Epub 2018 Nov 19. PMID: 30467064.
75. Walsh EE, et al. 2020. Safety and
immunogenicity of two RNA-based Covid-19
vaccine candidates. N Engl J Med. 383(25):
2439–2450.
doi: 10.1056/NEJMoa2027906. Epub 2020 Oct
14. PMID: 33053279; PMCID: PMC7583697.
76. Polack FP, et al. 2020. C4591001 Clinical Trial
Group. Safety and efficacy of the BNT162b2
mRNA Covid-19 vaccine. N Engl J Med.
383(27):2603–2615.
doi: 10.1056/NEJMoa2034577. Epub 2020 Dec
10. PMID: 33301246; PMCID: PMC7745181.
77. Johns Hopkins University Coronavirus
Resource Center. 2020. COVID-19 dashboard
by the Center for Systems Science and
Engineering (CSSE) at Johns Hopkins
University.
https://coronavirus.jhu.edu/map.html.
78. Khan T, Agnihotri K, Tripathi A, Mukherjee S,
Agnihotri N, Gupta G. 2020. COVID-19: A
worldwide, zoonotic, pandemic outbreak.
Altern Ther Health Med. 26(S2):56–64. PMID:
32412918
79. World Health Organization. COVID-19: Global
literature on coronavirus disease.
https://search.bvsalud.org/global-literature-on-
novel-coronavirus-2019-ncov
80. Statista. Coronavirus (COVID-19) death rate in
countries with confirmed deaths and over 1,000
reported cases.
https://www.statista.com/statistics/1105914/cor
onavirus-death-rates-worldwide/
81. Poon, L.L.M., Peiris, M. 2020. Emergence of a
novel human coronavirus threatening human
health. Nat Med 26, 317–319.
https://doi.org/10.1038/s41591-020-0796-5
82. Galloway SE, Paul P, MacCannell DR, et al.
2021. Emergence of SARS-CoV-2 B.1.1.7
Lineage — United States, December 29, 2020–
January 12, 2021. MMWR 70(3):95–99.
https://www.cdc.gov/mmwr/volumes/70/wr/m
m7003e2.htm
83. Harcourt J, Tamin A, Lu X, et al. 2020. Severe
Acute Respiratory Syndrome coronavirus 2
from patient with coronavirus disease, United
States. Emerging Infectious Diseases. 26(6):
1266–1273. doi:10.3201/eid2606.200516.
84. Tinari S. 2021. The EMA COVID-19 data leak,
and what it tells us about mRNA instability.
BMJ 372:n627 doi:10.1136/bmj.n627
85. Ioannidis, J.P. (2021), Reconciling estimates of
global spread and infection fatality rates of
COVID‐19: An overview of systematic
evaluations. Eur J Clin Invest. Accepted Author
Manuscript e13554.
https://doi.org/10.1111/eci.13554
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
123
86. Government of the United Kingdom.
S1182_SPI-M-O_Summary_of_modelling_of_
easing_roadmap_step_2_restrictions.pdf
87. Corbett, K.S., Edwards, D.K., Leist, S.R. et al.
2020. SARS-CoV-2 mRNA vaccine design
enabled by prototype pathogen preparedness.
Nature 586, 567–571.
https://doi.org/10.1038/s41586-020-2622-0.
88. Centers for Disease Control and Prevention.
2021. COVID-19 vaccines for people with
allergies.
https://www.cdc.gov/coronavirus/2019-
ncov/vaccines/recommendations/specific-
groups/allergies.html
89. Noh J, Danuser G. 2021. Estimation of the
fraction of COVID-19 infected people in U.S.
states and countries worldwide. PLoS ONE
16(2): e0246772.
https://doi.org/10.1371/journal.pone.0246772
90. Alroy KA, et al. 2021. Population-based
estimates of coronavirus disease 2019 (COVID-
19)–like illness, COVID-19 illness, and rates of
case ascertainment, hospitalizations, and
deaths—Noninstitutionalized New York City
residents, March–April 2020. Clinical
Infectious Diseases. 2021;ciab038.
https://doi.org/10.1093/cid/ciab038
7
Supplementary Materials
Supplementary Figure 1: Injection rates in each
age group in the general population compared to
the total AE VAERS reports (above) and total
SAE VAERS reports (below)
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
124
Supplementary Table 1: The true deletions shown in the context of all missing data. The new VAERS
IDs assigned to the redundant entries are also shown.
Count VAERS ID m issing New VAERS ID True deletions DIED class ification (B&A) Adverse Event Deleted from date
1 918723 N TRUE Y(Location: foreign) Death 1/7/21
2 923149 N TRUE Y(Location: foreign) Death 1/7/21
3 930386 N TRUE Y Death 1/15/21
4 930418 N TRUE Y Death 1/15/21
5 934963 N TRUE Y Death 1/15/21
6 937985 N TRUE Y(Location: foreign) Death 1/15/21
7 940950 N TRUE Y Death 1/15/21
8 940954 930466 NA Y/Y Death 1/15/21
9 944273 N TRUE Y Death 1/15/21
10 944385 N TRUE Y Death 1/22/21
11 944659 944641 TRUE Y/Y Death 1/15/21
12 946097 935767 NA Y/Y Death 1/15/21
13 947974 940955 NA Y/Y Death 1/22/21
14 949547 945253 NA Y/Y Death 1/22/21
15 951960 985715 NA Y/Y Death 1/29/21
16 955878 N TRUE Y Death 1/22/21
17 957321 N TRUE Y(Location: foreign) Death 6/11/ 21
18 960437 N TRUE Y Death 1/22/21
19 964729 1329449 NA Y/ NA Death 1/29/21
20 964956 962940 NA Y/Y Death 1/29/21
21 966236 Dead in 30 mins TRUE Y Death 1/29/21
22 970044 950533 NA Y /NA Death 1/29/21
23 970139 950441 NA Y/Y Death 1/29/21
24 970161 ITP? TRUE Y Death 1/29/21
25 971561 962325 NA Y/Y Death 1/29/21
26 971800 921768 NA Y/Y Death 1/29/21
27 978872 971969 NA Y/Y Death 2/4/21
28 982778 935815 NA Y/Y Death 1/29/21
29 983482 978959 NA Y Death 2/4/21
30 999818 N TRUE Y(Location: foreign) Death 2/12/ 21
31 1000669 986901 NA Y/Y Death 2/4/21
32 1000910 977186 NA Error: Wrong Patient (documentation in EMR) Unevaluable 2/ 4/21
33 1004651 N TRUE Y Death 2/18/21
34 1011588 985527 NA Y/NA Death 2/18/21
35 1017127 989006 NA Y/Y Death 2/12/21
36 1020144 994544 NA Y/Y Death 2/12/21
37 1024103 N TRUE Y N o death 2/12/21
38 1024731 1024592 NA Y/Y Death 2/12/21
39 1045540 939050 NA Y/Y Death 4/1/21
40 1048687 N TRUE Y Cerebrovascular Accident 3/ 5/21
41 1051447 Litigation request TRUE Y Death 3/11/21
42 1064933 TRUE Y(Location: foreign) Death 8/6/21
43 1074247 N TRUE Y Death (2 y/o) 4/1/21
44 1076914 N TRUE Y Death 3/19/21
45 1102077 1090801 NA Y/Y Death 3/19/21
46 1108447 1145662? ( JJ?P?) N A Y/Y Death 4/1/21
47 1108969 1096497 NA Y/Y Death 3/19/21
48 1113963 1084036 NA Y/Y SARS -CoV-2 3/19/ 21
49 1122171 1084419/1126060 NA Y /Y Death 4/1/21
50 1131199 1037207 NA Y/Y Death 4/1/21
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
125
Supplementary Table 1 continued
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
126
Supplementary Table 1 continued
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
127
Supplementary Table 1 continued
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
128
Supplementary Table 2: Table using Under-Reporting Factor (URF) conversion (30x) to demonstrate
suggested actual numbers of AEs rather than simply reported values in VAERS.
Data source: VAERS/Analysis: Steve Kirsch, Dr. Jessica Rose
Unrelated events (blue): The goal for symptoms like metal poisoning, hepatitis, and otitis media (shown
in blue) is to look for the propensity to over-report this year. If this was just over reporting we’d see a rate
increase for these symptoms that are unrelated to the vaccines and are not comorbidities.
Pre-existing comorbidities (green): These conditions like diabetes and cancer in the table above increase
simply because of the increased number of people filing reports in 2021.
Symptoms: For all symptoms (Deaths and others), we limited the search to 20-60-year-olds since these
people are less noisy with respect to symptoms and younger people aren’t yet vaccinated [21].
Adverse Event (AE) Observe d AE 2021 (N) Numbe r AE (2015-20 19)
Expected
(Average/year)
Incide nce Rate (AE) (N/Average per year) URF adjusted (OBS*31)
Metal poisoning 2.0 47.0 9 .4 0.2 62.0
Otitis media 48.0 255. 0 51.0 0.9 1, 488.0
Hepatitis 3 31.0 1,457 .0 291. 4 1.1 10,2 61.0
Bursitis 189 .0 395.0 79. 0 2 .4 5,85 9.0
Conjunct ivitis 17 2.0 278.0 55 .6 3 .1 5,3 32.0
Caesarean section 38 .0 97.0 1 9.4 2.0 1,1 78.0
Wart 1.0 7.0 1.4 0.7 31.0
Rotator cuff syndrome 55.0 148. 0 29.6 1.9 1, 705.0
Breech delive ry 0.0 3.0 0.6 0 .0 0.0
Cancer 31 .0 132.0 26 .4 1 .2 961. 0
Diabetes 1 55.0 284.0 5 6.8 2.7 4,8 05.0
Obesity 14. 0 9.0 1.8 7.8 43 4.0
Lyme disease 42.0 53 .0 10.6 4.0 1 ,302. 0
Abortion Spontaneous 707.0 90.0 18.0 39.3 21 ,917. 0
Anaphylactic Reacti on 1,503 .0 2 04.0 40 .8 36. 8 4 6,59 3.0
Aphasia (inability to talk) 1,184 .0 5 5.0 11. 0 107 .6 36,7 04.0
Appendi citis 4 33.0 11.0 2. 2 196.8 1 3,42 3.0
Bell’s Palsy 2,637 .0 1 33.0 26 .6 99. 1 8 1,74 7.0
Blindness 72 3.0 86.0 17 .2 42. 0 2 2,413 .0
Cardiac arrest 719.0 14.0 2.8 2 56.8 2 2,289 .0
Chills 6 1,97 2.0 4,7 25.0 9 45.0 6 5.6 1,92 1,132 .0
Cough 9,63 7.0 1,0 02.0 20 0.4 4 8.1 298,747 .0
Deafness 1,02 2.0 1 17.0 2 3.4 43 .7 31,6 82.0
Death 6,6 39.0 90 .0 18.0 368.8 2 05,8 09.0
Deep vein throm bosis 1,47 3.0 14. 0 2.8 52 6.1 45 ,663. 0
Depression 50 3.0 48 8.0 97 .6 5 .2 15,59 3.0
Diarrhoea 13 ,495 .0 6 ,262 .0 1,25 2.4 10.8 418,3 45.0
Dyspnoea (difficulty breathi ng) 20,6 74.0 19 4.0 3 8.8 5 32.8 64 0,89 4.0
Dysstasia (difficult y standing) 1,349 .0 1 33.0 2 6.6 50 .7 4 1,81 9.0
Fatigue 61,9 00.0 4, 575. 0 915.0 67.7 1,9 18,9 00.0
Guillain-Barre syndrome (GBS) 44 8.0 378 .0 75.6 5.9 13, 888. 0
Headache 73, 565.0 6,23 1.0 1,2 46.2 59 .0 2,280 ,515. 0
Herpes zoster 4 ,807. 0 700. 0 140.0 34.3 149,0 17.0
Insulin resistance 6.0 6 .0 1.2 5.0 1 86.0
Multipl e organ dysfunction syndrome 26.0 37.0 7 .4 3.5 80 6.0
Myalgia 17, 047.0 3 ,208 .0 641. 6 26.6 528,4 57.0
Myocarditis 6 71.0 73.0 1 4.6 46 .0 20,8 01.0
Neuropathy 13 3.0 19 5.0 39. 0 3 .4 4,12 3.0
Paraesthesia 9,86 0.0 2,4 40.0 4 88.0 2 0.2 305,660 .0
Paralysis 179.0 4 11.0 8 2.2 2.2 5,54 9.0
Parkinson’s disease 26 .0 5 .0 1.0 26.0 8 06.0
Peric arditis 447 .0 49.0 9.8 45.6 13 ,857 .0
Pruritus 1 8,103 .0 11,2 50.0 2, 250.0 8. 0 5 61,1 93.0
Pulmonary embolism 1,19 1.0 10 .0 2.0 5 95.5 36 ,921 .0
Seizure 2,3 62.0 4 31.0 86 .2 27 .4 7 3,22 2.0
Complet ed suicide 19.0 3.0 0 .6 31.7 58 9.0
Thrombosis 1,58 8.0 45 .0 9.0 1 76.4 49 ,228 .0
Tinnitus 6, 523.0 282.0 5 6.4 11 5.7 202 ,213 .0
Total 324,64 9.0 47 ,112.0 9 ,422 .4 3,75 8.3 10,0 64,11 9.0
Sci, Pub Health Pol, & Law
Critical appraisal of VAERS Pharmacovigilance — Oct. 2021
129
Supplementary Table 3: Table showing injected versus un-injected individuals in the context of
hospitalizations in Israel. Chart courtesy of Dr. Rafael Zioni. Data source: Israel Ministry of Health.
8
Author statement
Funding
This project was funded by donations from the
public to the Joshua Kuntz IPAK Research
Fellowship at the Institute for Pure and Applied
Knowledge (https://ipaknowledge.org/joshua-
kuntz-research-fellowship.php).
