Study Setting

We analyzed observational data from Clalit Health Services (CHS) in order to emulate a target trial of the effects of the BNT162b2 vaccine on a broad range of potential adverse events in a population without SARS-CoV-2 infection. CHS is the largest of four integrated payer–provider health care organizations that offer mandatory health care coverage in Israel. CHS insures approximately 52% of the population of Israel (>4.7 million of 9.0 million persons), and the CHS-insured population is approximately representative of the Israeli population at large.17 CHS directly provides outpatient care, and inpatient care is divided between CHS and out-of-network hospitals. CHS information systems are fully digitized and feed into a central data warehouse. Data regarding Covid-19, including the results of all SARS-CoV-2 polymerase-chain-reaction (PCR) tests, Covid-19 diagnoses and severity, and vaccinations, are collected centrally by the Israeli Ministry of Health and shared with each of the four national health care organizations daily.

This study was approved by the CHS institutional review board. The study was exempt from the requirement for informed consent.

Eligibility Criteria

Eligibility criteria included an age of 16 years or older, continuous membership in the health care organization for a full year, no previous SARS-CoV-2 infection, and no contact with the health care system in the previous 7 days (the latter criterion was included as an indicator of a health event not related to subsequent vaccination that could reduce the probability of receiving the vaccine). Because of difficulties in distinguishing the recoding of previous events from true new events, for each adverse event, persons with a previous diagnosis of that event were excluded.

As in our previous study of the effectiveness of the BNT162b2 vaccine,10 we also excluded persons from populations in which confounding could not be adequately addressed — long-term care facility residents, persons confined to their homes for medical reasons, health care workers, and persons for whom data on body-mass index or residential area were missing (missing data for these variables are rare in the CHS data). A complete definition of the study variables is included in Table S1 in the Supplementary Appendix, available with the full text of this article at NEJM.org.

Study Design and Oversight

The target trial for this study would assign eligible persons to either vaccination or no vaccination. To emulate this trial, on each day from the beginning of the vaccination campaign in Israel (December 20, 2020) until the end of the study period (May 24, 2021), eligible persons who were vaccinated on that day were matched to eligible controls who had not been previously vaccinated. Since the matching process each day considered only information available on or before that day (and was thus unaffected by later vaccinations or SARS-CoV-2 infections), unvaccinated persons matched on a given day could be vaccinated on a future date, and on that future date they could become newly eligible for inclusion in the study as a vaccinated person.

In an attempt to emulate randomized assignment, vaccinated persons and unvaccinated controls were exactly matched on a set of baseline variables that were deemed to be potential confounders according to domain expertise — namely, variables that were potentially related to vaccination and to a tendency toward the development of a broad set of adverse clinical conditions. These matching criteria included the sociodemographic variables of age (categorized into 2-year age groups), sex (male or female), place of residence (at city- or town-level granularity), socioeconomic status (divided into seven categories), and population sector (general Jewish, Arab, or ultra-Orthodox Jewish). In addition, the matching criteria included clinical factors to account for general clinical condition and disease load, including the number of preexisting chronic conditions (those considered to be risk factors for severe Covid-19 by the Centers for Disease Control and Prevention [CDC] as of December 20, 2020,18 divided into four categories), the number of diagnoses documented in outpatient visits in the year before the index date (categorized into deciles within each age group), and pregnancy status.

All the authors designed the study and critically reviewed the manuscript. The first three authors collected and analyzed the data. A subgroup of the authors wrote the manuscript. The last author vouches for the accuracy and completeness of the data and for the fidelity of the study to the protocol. There was no commercial funding for this study, and no confidentiality agreements were in place.

Adverse Events of Interest

The set of potential adverse events for the target trial was drawn from several relevant sources, including the VAERS, BEST, and SPEAC frameworks, information provided by the vaccine manufacturer, and relevant scientific publications. We cast a wide net to capture a broad range of clinically meaningful short- and medium-term potential adverse events that would be likely to be documented in the electronic health record. Accordingly, mild adverse events such as fever, malaise, and local injection-site reactions were not included in this study. The study included 42 days of follow-up, which provided 21 days of follow-up after each of the first and second vaccine doses. A total of 42 days was deemed to be sufficient for identifying medium-term adverse events, without being so long as to dilute the incidence of short-term adverse events. Similarly, adverse events that could not plausibly be diagnosed within 42 days (e.g., chronic autoimmune disease) were not included.

Adverse events were defined according to diagnostic codes and short free-text phrases that accompany diagnoses in the CHS database. A complete list of the study outcomes (adverse events) and their definitions is provided in Table S2.

For each adverse event, persons were followed from the day of matching (time zero of follow-up) until the earliest of one of the following: documentation of the adverse event, 42 days, the end of the study calendar period, or death. We also ended the follow-up of a matched pair when the unvaccinated control received the first dose of vaccine or when either member of the matched pair received a diagnosis of SARS-CoV-2 infection.

Risks of SARS-CoV-2 Infection

To place the magnitude of the adverse effects of the vaccine in context, we also estimated the effects of SARS-CoV-2 infection on these same adverse events during the 42 days after diagnosis. We used the same design as the one that we used to study the adverse effects of vaccination, except that the analysis period started at the beginning of the Covid-19 pandemic in Israel (March 1, 2020) and persons who had had recent contact with the health care system were not excluded (because such contact may be expected in the days before diagnosis).

Each day in this SARS-CoV-2 analysis, persons with a new diagnosis of SARS-CoV-2 infection were matched to controls who were not previously infected. As in the vaccine safety analysis, persons could become infected with SARS-CoV-2 after they were already matched as controls on a previous day, in which case their data would be censored from the control group (along with their matched SARS-CoV-2–infected person) and they could then be included in the group of SARS-CoV-2–infected persons with a newly matched control. Follow-up of each matched pair started from the date of the positive PCR test result of the infected member and ended in an analogous manner to the main vaccination analysis, this time ending when the control member was infected or when either of the persons in the matched pair was vaccinated.

The effects of vaccination and of SARS-CoV-2 infection were estimated with different cohorts. Thus, they should be treated as separate sets of results rather than directly compared.

Statistical Analysis

Because a large proportion of the unvaccinated controls were vaccinated during the follow-up period, we opted to estimate the observational analogue of the per-protocol effect if all unvaccinated persons had remained unvaccinated during the follow-up. To do so, we censored data on the matched pair if and when the control member was vaccinated. Persons who were first matched as unvaccinated controls and then became vaccinated during the study period could be included again as vaccinated persons with a new matched control. The same procedure was followed in the SARS-CoV-2 infection analysis (i.e., persons who were first matched as uninfected controls and then became infected during the study period could be included again as infected persons with a new matched control).

We used the Kaplan–Meier estimator19 to construct cumulative incidence curves and to estimate the risk of each adverse event after 42 days in each group. The risks were compared with ratios and differences (per 100,000 persons).

In the vaccination analysis, so as not to attribute complications arising from SARS-CoV-2 infection to the vaccination (or lack thereof), we also censored data on the matched pair if and when either member received a diagnosis of SARS-CoV-2 infection. Similarly, in the SARS-CoV-2 infection analysis, we censored data on the matched pair if and when either member was vaccinated. Additional details are provided in the Supplementary Methods 1 section in the Supplementary Appendix.

We calculated confidence intervals using the nonparametric percentile bootstrap method with 500 repetitions. As is standard practice for studies of safety outcomes, no adjustment for multiple comparisons was performed. Analyses were performed with the use of R software, version 4.0.4.