There is has been a great deal of interest in data coming out of Israel regarding possible rapidly waning immunity of the Pfizer-BioNTech COVID-19 vaccine. This news, combined with the bad news surrounding the virus delta variant represents very discouraging news. Some have concluded that the data from Israel’s health ministry indicate that the vaccine is currently only 39% effective at preventing transmission of the virus. And even more alarming is the apparent rate at which the efficacy of the vaccine appears to decline in a period of only a few months. Their data suggest that people receiving the vaccine in January 2021 are currently only experiencing 16% efficacy in protection from infection.
I suspect there are some methodological flaws in the conclusions being drawn for their data. It is clear breakthrough infections are occurring, and probably at a higher rate than in earlier variants of the virus. But the dramatic decline in efficacy is likely a mis-interpretation of the data. Figure 1 shows one of the tables from Isreal’s health ministry.
https://www.gov.il/BlobFolder/reports/vaccine-efficacy-safety-follow-up-committee/he/files_publications_corona_two-dose-vaccination-data.pdf
Varying Length of Exposure to Re-infection
The first possible data mis-interpretation involves the varying periods of time during which vaccinated people have faced potential infection. It appears from the graphs provided by the ministry of health that total counts of the number of breakthrough infections since vaccination may have been used as the numerator for efficacy, while the total number of vaccinations during that period were used as denominators. The problem with this approach is that people who were vaccinated in January have had a much longer period during which they face potential exposures that could result in infection. So for example, those people vaccinated in January, have had 6 months during which infection might occur, while those vaccinated in July presumably only have had 30 days of potential infection. In such a situation it is no surprise that people who got vaccinated 180 days ago, have experienced more infections than those who were vaccinated 30 days ago.
So to make an equal comparison, the time periods need to be equal. One way of doing this would be to just compare the number of cases during the most recent time period available for all cohorts. So if we just look at the last blue circles representing data from the last week for which data was collected you might get a more equal comparison. Looking at the right hand graph in figure 1, we still see the January vaccine cohort experiencing a higher breakthrough infection rate during the last week (74/100,000) than say the May cohort with a weekly rate of only 27/100,000.
Figure 1 illustrates another important consideration when looking at infection counts. The pattern across all cohorts shows significant changes between the January period and say the May and early June period. All cohorts January through May, show a marked reduction in breakthrough infections in that period. Understanding breakthrough infections (or any infections for that matter) requires a way of estimating the likelihood of exposure. Where there is no on-going infections, say for example in New Zealand, there are no “breakthrough” infections. Looking at the number of active cases in the population provides an estimate of the likelyhood of exposure to the virus.
If you compare the curve in figure 2 between January and July, you will see a clear correlation with the size of the blue dots in figure 1. In January when there were an estimated 75,000 active cases in Israel we see higher numbers of breakthrough infections, while when active cases were very low, we see very few breakthrough infections. Then in July when active cases start to rise again, we see larger number of breakthrough infections. This data make clear controlling community spread of infection has a much larger impact on breakthrough infections, than possible changes in immunity over time after vaccination.
Efficacy Calculations Require a Valid Comparison group
Vaccine Efficacy is generally calculated based on the number of cases occurring in the vaccinated group compared to a similar group not receiving the vaccination. This is how the original trials of the vaccine’s determined their efficacy. The Israeli data does not appear to make any attempt to compare breakthrough infections with the number of infections occurring in non-vaccinated populations. But instead compares the number of breakthrough infections over time in different vaccination cohorts. This is interesting data, but may not deliver valid efficacy figures. Nor does the data appear to attempt to control for other known risk factors such as underlying conditions or behavior differences. There is a limited stratification by age, as the report provides one table for over 60+ (figure 3) and a second table (Figure 1) vaccinated subjects 15-59. It is interesting to note that comparing only the July period for for the 60+ group, the differences in breakthroughs during the July period are far less striking.
| Month of Vaccination | July Breakthrough infections per 100,000 |
| May | 41 |
| April | 45 |
| March | 41 |
| February | 58 |
| January | 115 |
Looking only at breakthroughs in the 60+ population for the the month of July, the variabily by month of vaccine is far less consistent than the data in figure 1. Only the January cohort shows a markedly larger breakthrough rate. When controlling at least partially for age, the data does not show such a strong and consistent decline in vaccine protection over the 5 month period of study. By lumping 15-59 into one analysis shown in the Figure 1, we are likely hiding an artifact of the vaccine rollout process that targeted older and high risk groups earlier in the vaccine campaign, resulting in younger and healthier subjects being disproportionally represented in the later vaccination cohort. Hence at least partially accounting for the change in breakthrough rates between earlier and later vaccination cohorts.
Lastly, and most difficult to control for, is the issue of differences in human behavior. When evaluating data like shown in the Health Minstry report we need to ask if there are any important behavioral differences between the earlier and later vaccination groups. Some possible considerations include:
- Do long vaccinated groups begin to take more risks after a few months?
- Do people with fewer risk factors delay vaccination while high risk people seek earlier vaccination?
- How does the gradual increase in delta variant in different segments of the community impact breakthrough rates?
The Israeli data clearly shows that breakthrough infections are occurring. Not surprisingly, they occur at a higher rate when there are more active cases of the infection in the community. None of this should be too surprising. Since the first vaccine was approved for emergency use, we learned that they were highly effective at preventing severe COVID-19, but less effective at preventing mild symptomatic infections. Most of the trials did not even attempt to address the efficacy of preventing asymptomatic infections. This was not some horrible oversight in vaccine development, but the result of the clear pressure to bring a vaccine to market. Measuring efficacy in preventing asymptomatic infections would have required larger and longer trials. Something we as a society could not afford.
Coronavirus Epidemiology 