The goal of pharmacovigilance is early detection of adverse events (AEs) and appropriate and timely response in order to minimize negative effects to the health of individuals. There is a number of national and international postmarketing surveillance systems, including the VigiBase [
The Vaccine Adverse Event Reporting System (VAERS) [
These complex activities can be tedious and long lasting for regulatory authority scientists, primarily due to the fact that large parts of the data come in free-text. This kind of format makes the act of performing efficient systematic analysis difficult. To alleviate the situation in VAE safety detection and prediction, advanced text-mining, and other techniques are employed for feature extraction, semantics, and rule deduction (see [
While extensive work exists that copes with similar challenges regarding the general AE study and characterization, data-driven approaches have been shown to be powerful in AE detection and prediction [
In this regard, rational mechanism based assessment of pharmacovigilance statistics plays important role for vaccine safety scientists [
Additionally, despite the fact that vaccine immunization mechanisms are at large different than the main therapeutic-intervention biology in the presence of a disease, a number of cases have indicated the possibility of vaccine-drug interactions [
In this work, we analyze VAERS to provide examples of such computational challenges and highlight the importance of structuring VAE data (Table
Computational challenges | Description | Type |
---|---|---|
Reported VAE content | VAE reporting systems may also contain cases for which it is unclear whether a vaccine caused the VAE. Also follow-up is not always possible. VAE data alone cannot be used to determine a cause-effect relationship between a vaccination and an AE. | Qualitative |
Large parts of the data come in free text | Examples include narratives regarding patient medications, laboratory results, or disease history. Advanced text mining or other techniques can be employed for feature extraction, semantics, and rule deduction. | |
| ||
Mining unstructured content | One way to structure VAE data is by mapping content to organized dictionaries and/or hierarchies of therapeutic agents (e.g., vaccines and drugs) or phenotypic manifestations (e.g., diseases, medical conditions, symptoms, side-effects, and reactions). These tasks can be complicated, affected by several factors such as the nonstandard nature of the used nomenclature (e.g., country specific names), nonrelevant content, quality of the entity recognition method used, completeness of the underlying dictionary/hierarchy, annotation coverage, and appropriate representation/detection of relationships. | Quantitative |
Automated signal detection | While disproportionality metrics are utilized as the main signal detection standard, there is no sufficient (or universal) definition of a threshold for identified signal strength above which a potential relationship should be considered interesting for further investigation. Also, detected signals may sometimes refer to false positive associations. |
Socioeconomic challenges | Description | Pharmacovigilance aspect | |||
---|---|---|---|---|---|
Detection | Assessment / understanding | Prevention | Communication | ||
VAE reporting | VAERS contains only VAEs and symptom incidence is not normalized with respect to overall population vaccine consumption data. | X | X | X | |
| |||||
Vaccine development | Cancer vaccine therapeutics and vaccination of adolescents and adults is an important part of current research focus and clinical trial activities. | X | |||
| |||||
Cost management | While it is beneficial for the healthcare systems to prevent unnecessary or avoidable costs, political, organizational and logistical challenges may significantly hinder the delivery of large-scale vaccine administration programs. | X | X | ||
| |||||
Digital services | While use and development of digital services can promote the coordination of healthcare stakeholders, systemize real world data collection, help raise awareness, and empower both patient and physician engagement in immunization practices, relevant mobile phone services that are provided currently are largely maintained by authorities, primarily aiming to reach mainly health professionals. | X | X | X | |
| |||||
Collaborative health policies | Shared and better-informed decision-making is key for improving international efforts in harmonizing worldwide vaccine management and information. | X | X | X | |
| |||||
Public opinion | VAE data may contain biases and may be influenced by public response to news and media attention. | X | X | ||
| |||||
Vaccine hesitancy | While key part of vaccine information relates to safety and precaution issues, the easy spread of news, lack in education, and reduced disease infection rates have contributed to increased perception of vaccine-induced risks. It becomes increasingly necessary for voluntary vaccination programs not only to communicate these risks but also to emphasize the benefits of vaccination for the population in order to incentivize and promote community protection. | X | X |
To understand intricacies underlying VAE data, we reviewed VAERS content. First we annotate drug mentioning in VAE cases and, then, follow a dual analysis approach: We explore the extent of drug interference in VAEs and also assess the prevalence of reactions in those cases We evaluate whether it is possible to automatically generate comprehensive vaccine safety profiles from these data
To perform both of these tasks we first had to expand VAE content with drug and/or molecular information—Figure
We extracted VAE data from VAERS and drug and molecular information from DrugBank [ Cases with drugs known to interact with each other (DDIs) Cases with drugs known to affect the therapeutic efficacy, or the VAE risk, or severity of vaccinations (DVIs) Cases with potential interactions between drugs due to perturbation of the same targets (DTIs) Cases with drugs sharing the same metabolizing enzymes (DMIs)
To identify drugs mentioned in VAERS, we followed a previously employed approach [
To explore the relative association of symptoms and outcomes to different VAE-sets we employed two main computational techniques.
Proportional reporting ratio (PRR): the PRR metric is defined as the value of
VAE cases | | | Totals |
---|---|---|---|
| | | |
| | | |
Totals | | | |
In all cases, Fisher’s exact test (two tailed) was used to determine the statistical significance of each observation. Last, we defined minimum occurrence in at least ten VAEs as reasonable threshold to consider a relationship meaningful.
This work utilized PostgreSQL 9.6 for storage, Python for calculations, and Java for additional programming tasks.
We processed 607223 VAE reports from VAERS that contained 218 vaccine-names and 10169 symptoms (Additional File
DrugBank contained mainly small molecule therapeutics (i.e., low molecular weight drugs produced by chemical synthesis), but also agents manufactured in or extracted or semisynthesized from biological sources. These included medical agents (e.g.,
Using this dictionary, we matched drug names to VAERS medication narratives. We noticed that some were quite noisy (e.g., containing abbreviations, information about manufacturer, dosage, medication schedule, patient history, dramatic complaints, etc.), while others mentioned cosmetic or nutrition agents and other supplements. Characteristically we found potentially 13732 such VAEs referring to (multi-) vitamin use, just by annotating the mentioning of “VIT” in a narrative. Overall, by matching 77314 medication narratives (81%), we successfully annotated 102487 (16.9%) VAEs. Of those, 98963 (96.6%) linked to 1491 (46.3%) approved drugs.
A large proportion of VAEs (37.5%) had no medication narratives, while some were not informative. For example, the top five most frequently occurring phrases included “NONE” (103490 VAEs), “NO OTHER MEDICATIONS” (72735 VAEs), “UNKNOWN” (30951), “UNK” (23455 VAEs), and “CONCOMITANT DRUG(S) NOT REPORTED” (4428 VAEs). In our mapping approach we did not account for typographical and spelling errors that appeared in some narratives (e.g., “%DEXAMETHAZON%,” “AVENDOL, TREZEDONE AND DESIPREANINE,” or “LEXAPOR TRAZADONE”). We also did not consider advanced regular expressions, types of drug classes, or semantics that would help in few cases to avoid both false negatives and false positives. For example, the phrase “LATANOPROST0.005% EYE DROPS” was falsely not mapped, while
However, VAERS does not contain only medication narratives. Because mining unstructured free text can be challenging, especially regarding biomedical nomenclature [
We then focused on VAE symptoms—VAERS contained 10169 symptoms coded in terms coming from 23 different MedDRA versions. Most symptoms (60.8%) appeared in <10 VAEs (Figure
Furthermore, a large amount of VAEs refer to elderly (Figure
Our vaccine content indicated that, of the 98963 VAEs that had been mapped to approved drugs, more than half (55%) linked to multiple drugs (>1; vitamins not considered). Of these 54454 polypharmacy VAEs (8.97% of all VAEs), 6172 (11.3%) were serious and accounted for 8.1% of the total serious VAEs. Overall, 8620 reactions (84.8%) were reported with 76122 serious VAEs (Figure
We therefore investigated polypharmacy cases further and assessed the distribution of symptoms and of serious outcomes among VAEs with higher likelihood of drug interference. We defined those drug interference VAEs as polypharmacy cases that contained known DDIs, or potential DTI- or DMI-inferred interactions. We also included cases for which drug interference might not be attributed to polypharmacy alone and looked for VAEs mentioning at least one DVI drug.
In total, we identified 53899 such possible drug interference VAEs (8.88% of all VAERS): interestingly, this set contained 16202 VAEs with DVIs alone, already a significant proportion of VAERS (2.7%). The set contained also 38157 VAEs with DDIs and 7715 and 40052 VAEs with DTIs and DMIs, respectively. Notably, manifestation of serious outcomes is exacerbated among those VAEs (Figure
Moreover, we analyzed the 5460 symptoms mentioned in drug interference VAEs (Additional File
These results verified higher occurrence of drug-induced events in drug interference VAEs, but they also revealed a range of errors for the remaining set of VAEs that could be attributed to vaccine administration or to medical and therapeutic procedures. We believe that, irrespective of whether it was iatrogenic or patient factors underlying those cases, their occurrence calls for improved immunization practices and raises the issue of education to highlight awareness for both medical personnel and patients.
Next, we sought to characterize the relationship between vaccinations and symptoms, as reported in VAEs. The dataset held 218 vaccination names for ninety vaccine types (Additional File
By filtering out nonsignificant associations, our analysis narrowed down the set by 91.5% and 79% with respect to the total candidate relationships and symptoms, correspondingly. Characteristically, ten vaccine types were mentioned in too few VAEs and had no significant associations. Our threshold of choice was maybe too strict, favoring thus confidence in cooccurrences with larger numbers of VAEs (Table
Totals | Unprocessed set | Profile summary |
| ||
Symptom-to-vaccine combos | 132093 | 11287 |
Vaccine types | 90 | 80 |
Symptoms | 10169 | 2133 |
| ||
Averages | Unprocessed set | Profile summary |
| ||
VAEs per vaccine type | 10614.2 | 11938.25 |
Symptoms per vaccine type | 1467.7 | 141.09 |
PRR score (symptom-vaccine type) | - | 11.37 |
% symptom occurrence per vaccine type | 0.3 | 1.34 |
We chose to validate our results by looking at the safety profile produced for BCG (Table
Reaction | Num of VAEs (total) | Num of VAEs (BCG) | PRR | %BCG’s VAEs |
---|---|---|---|---|
BOVINE TUBERCULOSIS | 15 | 14 | 20178.7 | 3.32542 |
TUBERCULOSIS | 32 | 12 | 864.8 | 2.85036 |
DYSURIA | 500 | 20 | 60.1 | 4.7 |
LYMPHADENITIS | 355 | 13 | 54.8 | 3.1 |
POLLAKIURIA | 416 | 14 | 50.2 | 3.3 |
RESPIRATORY RATE INCREASED | 566 | 17 | 44.6 | 4 |
INJECTION SITE ABSCESS | 1032 | 12 | 16.9 | 2.8 |
POLYMERASE CHAIN REACTION | 1344 | 13 | 14.1 | 3.1 |
HAEMOGLOBIN NORMAL | 1258 | 11 | 12.7 | 2.6 |
DEATH | 2766 | 20 | 10.5 | 4.8 |
HAEMATOCHEZIA | 2058 | 11 | 7.7 | 2.6 |
LYMPHADENOPATHY | 7759 | 33 | 6.2 | 7.8 |
PNEUMONIA | 3310 | 13 | 5.7 | 3.1 |
COUGH | 13116 | 36 | 3.9 | 8.6 |
LABORATORY TEST ABNORMAL | 6327 | 15 | 3.4 | 3.6 |
IRRITABILITY | 7757 | 12 | 2.2 | 2.9 |
INFECTION | 13014 | 19 | 2.1 | 4.5 |
CHILLS | 19150 | 24 | 1.8 | 5.7 |
DIARRHOEA | 16105 | 20 | 1.8 | 4.8 |
PYREXIA | 100453 | 107 | 1.5 | 25.4 |
VOMITING | 28847 | 30 | 1.5 | 7.1 |
Our results indicate that vaccines are overall safe—indeed, immunization is one of the most cost-effective public health interventions to date, saving millions of lives [
Such health effects can translate also into positive economic results, as vaccination can provide significant savings by avoiding direct and indirect costs associated with the treating of diseases and possible long-term disabilities [
While these developments represent potentially important consequences for healthcare systems and the health of citizens, they also encourage investing in research and development (Figure
Indeed, it is expected that major role in the future of vaccine pharmaceutics will play revenue potential from vaccination of adolescents and adults, as opposed to sales from the vaccination of children that drove this market in the past. This is somewhat reflected by VAERS age demographics (Figure
Expecting the returns of a long, risky, and expensive discovery process, industry drives big part of clinical trial development, while a variety of other stakeholders participate with the incentive to develop new, cheaper, and safer vaccines. Vaccinomics play a special role in this process, enabled by the widespread diffusion of high-throughput omics disciplines, technologies, and approaches in the field of vaccinology [
Another aspect influenced by management and administration policies is public opinion. One such example is the public concern caused by the 2009 swine flu vaccine shortage and its direct impact on vaccine safety perception. This is elegantly demonstrated by the peak in Google searches for “Vaccine safety” in October 2009 [
On the positive side, (pre-)school vaccine administrations are more due to government mandates and support, rather than result of economic or public opinion incentive. There are, however, considerable political, organizational, and logistical challenges to the delivery of such large scale programs. Challenges include funding, vaccine supply and distribution, staff capacity and workload, anxiety and distress to students, and consent and reach of parents [
Production and consumption of personalized health apps may be one way to enable such new collaborative models. Several studies in mobile use have demonstrated that active patient participation can benefit vaccination programs [
However, we find that this market has not yet reached its potential. Studies show that vaccination coverage in mobile apps follows neither the growth of media use nor the related advancement of technological features [
Education also plays important role—while digital technologies may serve well as a mechanism to empower users and increase participation in the immunization process, they have also revolutionized our ability to educate ourselves. Reasonably, key part of vaccine information relates to safety and precaution issues regarding contraindications and allergies. However, it becomes increasingly necessary to communicate the need to make vaccinations as planned, to all members involved in each society.
This is because several reasons exist that may have undermined vaccine importance. First, disease eradication occurring in some places may mask the cost-benefit relationship for an individual, family, or community. Then, vaccine credibility may have been weakened by the familiarization of the public with circumstantial profit-driven industry practices. Furthermore, this does not help adequately limit a dilemma that some doctors perhaps may often face: to take the responsibility that a vaccine will have no side effects, and this, regardless of the fact that it is not absolutely certain it will provide the desired immunity.
The answer is not univocal. Certainly safety concerns should be communicated, but not at the expense of how general immunization is perceived. Game theory models show that it is “herd immunity” rather than self-interest that can help outweigh the risk of infection through vaccination [
Vaccines have historically improved quality of life. Optimizing earlier capture of safety and error risks can help leverage vaccine value and provide higher levels of health quality. However, to accelerate modern pharmacovigilance insight requires strategies that are able to provide more mechanistic (causative) explanations of observed safety concerns [
VAERS is one such source of VAE observations, but its content must be dealt with caution when interpreted, as these data alone cannot be used to determine a cause-effect relationship between a vaccination and an AE [
In the context of this work, VAERS was used for hypothesis generation—we assessed the extent of polypharmacy-induced risks and found that prevalence of serious outcomes is higher in VAEs with more definitive risk of drug interference. This also suggests that many serious VAEs may be falsely attributed to vaccines.
Facilitating such data-driven techniques for broader analytics is one factor for determining strategies to improve safety [
As mobile technology continues to rapidly evolve, we expect that mobile apps offer the potential to improve the quality of information residing in immunization evaluation programs, facilitate harmonization between individuals, health care providers and public health systems, and may help reduce vaccine hesitancy—a hesitancy that may perhaps be attributed to several factors. Some of those include the fact that reduced disease infection rates have contributed to increased perception of vaccine-induced risks, the easy spread of news through modern media, and the lack of education about immunization, what vaccines are or how they work. In some ways, fear of disease became fear for the vaccine—some might say that vaccines have been the victims of their effectiveness.
Game theory models explain that this is an understandable behavior, reasonably driven by individual self-interest. They do, however, also provide “selfish” arguments towards performing the “altruistic” act of vaccination that governments should harvest. Voluntary vaccination programs should incentivize and promote community protection and highlight the expectation to save millions of lives. The economic cost estimate to this synergistic individual-population benefit plays also an important role to make the right decisions on vaccination policy.
Our work also calls for the development of more refined algorithms that will allow for novel data streams to be combined and mined. Big data play key role in this perspective, which have contributed and are expected to continue contributing toward facilitating the discovery, development, production, and delivery of more rationally designed vaccines and immunization practices [
In the future we plan to advance and automate our approach for reviewing VAERS and to systematically provide services for researchers and the public. We expect to benefit from updated drug, molecular, and VAERS content, as well as considering also information about foods, fruits, and nutritionals or supplements. To address data extremities we want to enhance our analysis with extended synonym dictionaries and ontologies and hierarchies for reaction categories and drug classes. Last, we also plan to expand our approach by testing against known vaccine and drug side effects, examine indications and subpopulation susceptibility, and investigate the influence of combinatorial drug and vaccine occurrences in the incidence of specific symptoms.
We envisage that our work will provide a broad understanding of the socioeconomic and computational challenges underlying vaccine pharmacovigilance, as well as an attractive framework for improving the performance of safety signal detection algorithms. We demonstrated that structuring AE data and integration of molecular information can potentially provide additional insight into existing approaches, but also an easy way to quickly and systematically produce safety hypotheses. Importantly, it enables a standardized approach to the development of more objective analytics and promotes public domain transparency. We find that key to any healthcare system stakeholder is the adoption of integrated safety assessment and interpretation strategies, not only to avoid adverse incidence and preventable costs, but importantly to accommodate opportunities for advancing community health, personal awareness, and quality of life.
Adverse event
Bacillus Calmette-Guerin
Common Terminology Criteria for Adverse Events
Drug-drug interaction
Drug-metabolizing enzyme interaction
Diphtheria and tetanus toxoids, pediatric
Drug-target interaction
Diphtheria and tetanus toxoids and pertussis vaccine
Drug-vaccine interaction
Human immunodeficiency virus
Human papilloma virus
Food and Drug Administration
FDA Adverse Event Reporting System
Medical Dictionary for Regulatory Activities
Ontology of Vaccine Adverse Events
Proportional reporting ratio
Vaccine adverse event
Vaccine Adverse Event Reporting System
World Health Organization.
The data used to support the findings of this study are included within the supplementary information files.
The authors declare that they have no competing interests.
Vasiliki Soldatou helped with data integration and analytics, Anastasios Soldatos reviewed socioeconomic aspects, and Theodoros Soldatos conceived and led the overall study; Vasiliki Soldatou, Anastasios Soldatos, and Theodoros Soldatos wrote the manuscript.
File name and format: Additional_File_1.txt. Title: Occurrence of reactions in VAERS. Description: Tab separated file containing the number of VAEs in which each reaction was reported.
File name and format: Additional_File_2.txt. Title: Symptom over-representation in drug interference VAEs. Description: Tab separated file containing the name of each reaction (event_symptom_reaction), the total number of VAE cases the reaction was reported in (event_count), the total number of cases that the reaction was reported in drug interference VAEs (event_count_in_drug_interference_vaes), respective contingency matrix 1 values (a, b, c, d), the respective PRR score and 95% confidence interval (prr, prr_95ci_min, prr_95ci_max), and the respective Fisher’s two-tailed test p-value and if it is <0.05 (pvalue_fisher_exact_test_two_tailed, significant_by_fisher_exact_test).
File name and format: Additional_File_3.txt. Title: Vaccine occurrence in VAEs. Description: Tab separated file listing the vaccination type code (vax_code), the number of VAE cases reporting this vaccine type (num_vaes_type), vaccination names reported in VAERS for each vaccine type (vax_name), and the number of VAEs that the respective vaccine name was reported in (num_vaes_name).
File name and format: Additional_File_4.txt. Title: Vaccine reaction profiles. Description: Tab separated file containing vaccine types (vax_type), number of VAEs that the vaccine type was mentioned in (num_vaes_vax_type), reaction names co-reported with the respective vaccine type (symptom_reaction_event), total number of cases that the reaction was mentioned in (num_vaes_symptom), number of VAE cases that the respective vaccine type and reaction were co-reported (num_vaes_vax_n_event), contingency matrix 1 values (a, b, c, d), respective PRR score and 95% confidence interval (prr, prr_95ci_min, prr_95ci_max), the respective Fisher’s two-tailed test p-value and if it is <0.05 (pvalue_fisher_exact_test_two_tailed, significant_by_fisher_exact_test), and percent (%) of the vaccine type’s VAEs that the reaction was co-reported (percent_event_in_vax_vaes).
File name and format: Additional_File_5.pdf. Title: Supplementary methods and figures. Description: Supplementary method details and Figures 1 and 2.