Effectiveness of COVID-19 vaccines against hospitalisation, death and infection over time in Aotearoa New Zealand: a retrospective cohort study

Between the confirmation of the first COVID-19 case in Aotearoa New Zealand and October 2023, there were over 2,470,435 cases, with 31,119 hospitalisations, 878 intensive care unit admissions and 4,849 deaths.¹ The risk of severe COVID-19 and mortality increases with age, relative socio-economic deprivation, disability status and comorbidity, and the age-adjusted COVID-19 mortality is higher among Māori and Pacific peoples compared to the general population.^2–5 At the outset, the Aotearoa New Zealand Government pursued a strategy focussed on suppressing the community spread of SARS-CoV-2 and ultimately achieved extremely low or zero COVID-19 incidence.³ Throughout the pandemic, the Government adjusted the approach to match the evolving virus, transitioning from elimination to mitigation and ultimately prioritising vaccination.⁶

The national medical regulatory agency granted provisional approval for using the Pfizer–BioNTech COVID-19 vaccine (Comirnaty) on 3 February 2021. The vaccine rollout commenced with priority groups (border and managed isolation and quarantine workers, their household contacts and families) on 20 February 2021.² From January 2022, all individuals aged 5 and older in Aotearoa New Zealand became eligible for vaccination. Additionally, in February 2023, vaccination eligibility was extended to certain infants as young as 6 months old.² Several additional vaccines were approved and administered to a limited segment of the vaccinated population. These included AstraZeneca (Vaxzevria, approved in July 2021), Janssen (July 2021), Novavax (Nuvaxovid, March 2022) and Bivalent mRNA-CVs (December 2022).²

Accurately evaluating vaccine effectiveness (VE) and subsequent VE waning (understanding when VE declines) helps to inform public health decision making during an evolving pandemic. Understanding the waning of VE after the first, second and booster vaccine doses can also help identify who is most likely to benefit from further booster doses or other vaccines or interventions to reduce risk. For instance, a previous United Kingdom (UK) study found that significant vaccine waning occurred 25 weeks after the second dose of the vaccine.⁷ This observational study, among others, contributed to the evidence that was used to design the UK COVID-19 vaccine booster programme.

Using data from the Ministry of Health’s national COVID-19 surveillance platform and building on our previous work using routinely collected data in Aotearoa New Zealand,^8–10 we evaluated the VE of COVID-19 vaccines in preventing COVID-19 outcomes (hospitalisation, mortality and infection) over time since vaccination.

Study design

A retrospective, whole-population, matched-cohort study was conducted to evaluate the effectiveness of COVID-19 vaccines and vaccine waning (i.e., the decline in VE) using the national data collections provided by the Ministry of Health in Aotearoa New Zealand.

Data sources

The data used in this study were sourced from the Ministry of Health and the Institute of Environmental Science and Research (ESR; Figure 1 and Appendix Table 1). Additional data were sourced from the 2018 Census, Department of Internal Affairs (DIA) and core data derived by Statistics New Zealand (Stats NZ), including full death dates and address notifications.

A National Health Index (NHI) number is a unique identifier assigned to individuals accessing healthcare services in Aotearoa New Zealand.¹¹ It serves as a comprehensive record for data linkage and demographic data, encompassing crucial information such as name, address, date of birth, gender, resident or citizenship status, place of birth, ethnicity and, if applicable, date of death.

The Eclair Clinical Data Repository is a national reporting application in Aotearoa New Zealand, created by the Data and Informatics team at the ESR.¹² It compiles COVID-19 test reports, both positive and negative, from various sources across the country. Initially designed for COVID-19 reporting, it was expanded to support eOrdering for polymerase chain reaction (PCR) test and rapid antigen test (RAT) recording via the Eclair RAT Reporting system. This application managed and shared testing data with the government, Eclair users and partners (including researchers).

The COVID-19 Immunisation Register (CIR) is an application based on the National Contact Tracing Solution platform with robust security and authorisation controls.¹³ It includes information for monitoring immunisation coverage and the progress of the immunisation campaign. The creation of the CIR in 2020 stemmed from the limitations of the existing National Immunisation Register, which hindered its suitability for promptly facilitating a COVID-19 vaccination rollout on a national scale. It records the immunisations people have received or chosen not to receive. The data collected were necessary for health management, public safety, healthcare planning, research, professional training, statistical reporting and government service enhancement. The COVID-19 vaccination data were released to the public by the Ministry of Health.¹⁴ This was updated weekly to include recent changes in the values of those vaccinated and/or boosted. The spreadsheet released by the Ministry of Health contains a breakdown by district health board (DHB), territorial authority, health service utilisation population, ethnicity, vaccine type and cumulative values. DHBs are used to determine the area of residence for individuals included in this study.

EpiSurv is a secure national reporting and data repository system in Aotearoa New Zealand that tracks notifiable diseases (significant public health risk).¹⁵ It is operated by ESR on behalf of the Ministry of Health and is utilised by public health units for reporting cases. EpiSurv gathers up-to-date information encompassing baseline patient characteristics, clinical presentation, determinants and inter-case relationships used for disease surveillance.

The National Minimum Dataset (NMDS) is a comprehensive repository of hospital discharge information in Aotearoa New Zealand, encompassing data from both public and private healthcare facilities for inpatients and day patients and containing public and private hospitalisation data from 1997 onwards.¹⁶

The Pharmaceutical Collection is a comprehensive data repository that manages community-dispensed pharmaceutical subsidies in Aotearoa New Zealand and contains over 469 million claims.¹⁷

The Mortality Collection serves as a data repository on causes of death established in Aotearoa New Zealand.¹⁸ It includes electronic death and stillbirth registrations, medical certificates, data from hospital discharges and other agencies.

Population

The vaccinated cohort included everyone who received the COVID-19 vaccine (Comirnaty) between 8 December 2020 and 28 February 2023 (end of study). Medsafe, the nation’s medical regulatory agency, granted provisional approval for the use of Comirnaty in Aotearoa New Zealand on 3 February 2021. The vaccine rollout commenced with priority groups on 20 February 2021.² Our start date, 8 December 2020, was when the first COVID-19 vaccine doses were administered outside clinical trials.¹⁹ This date accounts for individuals who might have received the vaccine and returned to Aotearoa New Zealand before its official licensure. The unvaccinated cohort was comprised of people who did not receive any dose of the COVID-19 vaccine during the study period (Figure 2).

We matched vaccinated individuals with unvaccinated individuals based on age, sex, ethnicity and DHB to reduce confounding over time. This resulted in improved precision of estimates of VE. Due to the high vaccination rates in adults, there was a small pool of potential unvaccinated matches for adult vaccine recipients.

Exposure to vaccination

We studied the first and second vaccine doses and boosters (first and second). Most of the Aotearoa New Zealand population received the Pfizer–BioNTech (Comirnaty) vaccine.² Vaccination status was ascertained from codes provided directly by the Ministry of Health up to 28 February 2023, the latest date of available records.

Exposure (vaccination) status was defined as time-varying, with an individual defined as exposed from the date of immunisation. Our primary comparison of interest was the time elapsed since receiving the vaccine compared to not having received one. The unvaccinated cohort consisted of individuals who did not receive the vaccine. Controls who had not received a COVID-19 vaccine by the end of the study period were matched 1:1 based on socio-demographic characteristics.

Outcomes

COVID-19 hospitalisation was defined as being admitted to the hospital within 14 days of a confirmed SARS-CoV-2 infection or having an International Classification of Disease-10 (ICD-10) code for COVID-19. COVID-19 hospitalisations were derived as whether someone had never (0) or ever (1) been hospitalised for COVID-19 (with any COVID-related ICD-10-AM code) within 14 days of a positive COVID-19 test, for positive tests up to 28 February 2023 (end of follow-up).⁴ They were coded as first, second and third hospitalisations.

COVID-19 death was defined as having COVID-19, an underlying ICD-10 cause of death recorded on the death certificate, or any cause of death within 28 days of a confirmed SARS-CoV-2 infection.¹

COVID-19 infection was defined as cases of COVID-19 with a positive SARS-CoV-2 test. COVID-19 infection was used as a secondary outcome to determine whether an individual had ever (1) or never (0) recorded a positive COVID-19 test result, either before (PCR) or after (RAT) 16 February 2022. This cutoff point marked the beginning of unsupervised self-testing. As a result, positive or negative test results may not have been officially recorded uniformly, potentially inflating the ratio of positive to negative tests beyond this point.⁴ On 6 March 2022, the COVID-19 response minister cautioned that the reported daily case numbers might significantly under-estimate the actual cases due to self-reporting with RATs and delays in reporting results, emphasising that COVID-19 hospitalisations are considered a more reliable tracker of the pandemic.²⁰

Covariates

Comorbidity and multimorbidity encompass the presence of multiple distinct health conditions within an individual and are associated with increased healthcare burden, reduced quality of life and poor health outcomes.²¹ Multimorbidity was assessed using the Pharmaceutical Prescribing Profile mortality risk index (P3 index) and the MultiMorbidity Measure index (M3 index).^22,23 Age in years, birth month/year and sex (0=men, 1=women) were obtained from the Ministry of Health. Age was divided into 16 age groups. Ethnicity was divided into Māori, Pacific peoples, European, Asian and Middle Eastern, Latin American or African (MELAA). Māori ethnicity was coded for those recorded as Māori only or Māori and at least one other ethnic group.

Statistical analysis

The process of 1:1 matching was conducted using the Reclin2 package,²⁴ a set of tools designed for probabilistic record linkage. It consisted of the following steps: Pairs of records were generated from vaccinated and unvaccinated cohorts using two blocking variables (age group and sex). The generated pairs of records were compared on a set of variables (ethnicity and DHB) in both datasets (vaccinated and unvaccinated cohorts). The pairs were scored using the expectation–maximisation algorithm to estimate the m- and u-probabilities for each of the linkage variables (ethnicity and DHB). Pairs with a high likelihood were selected, i.e., pairs that met a predetermined threshold (effectively matched according to the potential confounding variables). Finally, using the selected pairs, the final linked dataset was generated.²⁴ This process was conducted four times (first dose vs unvaccinated, second dose vs unvaccinated, first booster vs unvaccinated and second booster vs unvaccinated).

Baseline patient characteristics, such as age, sex, ethnicity, comorbidity and multimorbidity (M3 score and P3 score), infectiousness, level of susceptibility, immunity (hybrid, infection induced and vaccine induced) and location (DHBs), were presented as frequencies and percentages for the matched cohorts (vaccinated and unvaccinated).

For the vaccinated cohort(s), the index date was the date of vaccination. Each vaccinated individual was monitored from the index date until they either developed the outcome of interest or received the next vaccine dose, or until the end of the study (28 February 2023). Unvaccinated individuals were assigned a pseudo-vaccination date (index date) that matched their paired vaccinated counterparts. The unvaccinated cohort was then monitored from this index date until they either developed the outcome of interest or until the end of the study.

For each outcome (primary and secondary), we evaluated the VE of COVID-19 vaccines (first and second doses, first and second booster doses) using the Cox proportional-hazards model. For each outcome, we fitted a model at each time point during the follow-up period (e.g., 1, 2, 3, 4, 5 and 6 months). Estimating the overall VE is unnecessary and may be misleading, as the impact of COVID-19 vaccines on outcomes depends on the time since vaccination. Our approach aimed to elucidate the trajectory (if any) of waning COVID-19 VE.²⁵ The model was adjusted for multiple confounders through the process of matching/probabilistic linking and the addition of covariates, including age, sex, ethnicity, DHB, comorbidity and multimorbidity, and numeric variables derived from the agent-based model (infection-induced immunity, vaccine-induced immunity, hybrid immunity, infectiousness and level of susceptibility; Appendix Table 1).

Subgroup analyses were performed by age group, sex and ethnicity. The models were similar, but in situations where there were fewer cases, we concatenated the time-since-vaccination indicators (e.g., 1–3 and 4–6 months, etc.). For each outcome, we estimated an adjusted hazard ratio (aHR) and 95% confidence intervals (CI) for each time-since-vaccination interval in comparison to the unvaccinated group. The VE was estimated as one minus the aHR, scaled as a percentage (1 - aHR * 100). Statistical analyses were carried out using R/R Studio (version R-4.2.2).²⁶

Results

Baseline characteristics of the population

A total of 5,269,015 people were included in the analysis. The mean age was 38.8 (standard deviation [SD] =23.2, range = 0–113) years. Between 8 December 2020 and 28 February 2023, a total of 4,318,211 (82.0%) individuals received the COVID-19 vaccine first dose, 4,158,014 (78.9%) received the second dose, 2,737,890 (52.0%) received the first booster dose and 755,107 (14.3%) received the second booster dose of COVID-19 vaccine (Table 3, Appendix Table 2). During the follow-up period, 413,310 (9.5%) individuals aged 15 and above were unvaccinated. Vaccination rates across all doses were highest among older adults, females, Europeans, Asians and people resident in the Capital and Coast DHB. The median times between the first and second doses, the second dose and the first booster dose, and the first and second booster doses were 4 weeks, 5 months and 7 months, respectively (Table 2).

During the follow-up period, there were 1,248,548 recorded COVID-19 infections, 24,370 hospitalisations and 1,006 deaths within 28 days of a positive test for SARS-CoV-2. A total of 5,375 (0.4%) people had a second infection, and 26 had a third. Additionally, 0.7% (169) of hospitalised individuals were readmitted for a second time (Table 1).

The trajectory of the second booster dose VE over time

The estimates of VE against hospitalisation (VE_H) and infection (VE_I) are shown in Table 3. No deaths occurred in the vaccinated cohort after the second booster dose, so death was not included in the analysis. Against COVID-19 hospitalisation, VE_H was 81.8% (95% CI: 73.6–87.5) in the 1st month, decreased to 72.2% (95% CI: 58.5–81.4) in the 4th month and then further decreased to 49.0% (95% CI: 7.9–71.8) in the 6th month. In the 1st month, the VE against COVID-19 infection was 57.4% (95% CI: 48.4–64.7). By the 4th month, it had decreased to 25.7% (95% CI: 04.8–42.1), and in the 6th month, it further decreased to 9.9% (95% CI: -25.8–35.4).

Waning VE was observed across all sub-groups (Appendix Table 3). The second booster dose VE against COVID-19hospitalisation for the Māori population was 81.1% (95% CI: 46.0–93.4) in the 1st month, decreased to 51.9% (95% CI: 7.6–74.9) in the 2nd–3rd month and then further decreased to 36.6% (95% CI: -16.8–63.4) in the 4th–6th month. The second booster dose VE against COVID-19hospitalisation for Pacific peoples was 92.2% (95% CI: 36.6–98.9) in the 1st month and decreased to 56.7% (95% CI: -0.7–81.3) in the 4th to 6th month.

The trajectory of the first booster dose VE over time

For each outcome of interest, VE was highest in the 1st month post-vaccination, then waned over time (Table 4). Against hospitalisation, VE_H was 81.6% (95% CI: 75.6–86.1) in the 1st month, was 74.7% (95% CI: 68.2–79.9) in the 3rd month and decreased to 57.4% (95% CI: 45.8–66.6) by the 6th month. Against COVID-19 death, there was sustained protection over the follow-up period. The VE_D was 92.9% (95% CI: 82.1–97.2) in the 2nd month and was 87.2% (95% CI: 67.4–94.9) during months 5–6 post-vaccination.

Against infection confirmed with real-time polymerase chain reaction (RT-PCR), VE_I was 54.0% (95% CI: 38.8–65.4) in the 1st month and dramatically decreased to 20.0% (95% CI: -88.1–66.0) in the 3rd month. Against infection (determined by both RAT and RT-PCR), VE_I was 20.2% (95% CI: 17.7–22.7) in the 1st month and decreased to 18.9% (95% CI: 16.7–21.0) 2 months post-vaccination. Additional findings on VE by subgroup can be found in the supplemental material (Appendix Table 4).

The trajectory of the second dose VE over time

For each outcome, VE for the second dose was highest in the 1st month post-vaccination, then waned over time (Table 5). Against hospitalisation, VE_H was 92.9% (95% CI: 86.4–96.3) in the 1st month, decreased to 74.7% (95%CI: 63.2–82.6) in the 3rd month and was 72.5% (95% CI: 64.9–78.5) by the 6th month. Against death, there was sustained protection over the follow-up period. The VE_D was 87.3% (95% CI: 53.8–96.5) in the 3rd month and was 86.1% (95% CI: 50.6–96.1) by the 6th month post-vaccination. VE_I against COVID-19 infection confirmed with RT-PCR was 88.2% (95% CI: 85.3–90.5) in the 1st month and decreased to 40.6% (95% CI: 25.6–52.5) in the 5th month. Vaccine waning was similar across all the sub-groups (Appendix Table 5).

The trajectory of the first dose VE over time

The estimates of VE against the outcomes of interest and sub-group analysis are shown in Appendix Table 6. Against hospitalisation, VE_H was 69.6% (95% CI: 50.1–81.5) in the 1st month and increased to 88.5% (95% CI: 80.6–93.1) in the 2nd month. Against death, there was sustained protection over the follow-up period, a VE_D of 87.6% (95% CI: 38.9–97.5). The VE against infection (RT-PCR confirmed) was 63.2% (95% CI: 56.1–69.2) during the follow-up period (1 month).

Discussion

We found that VE against hospitalisation and mortality was most robust in the early post-vaccination period, with VE against hospitalisation waning over time. A moderate waning of VE against death was found. Against COVID-19 infection, there were declines in VE, most notably at 3 months post-vaccination.

Our findings were broadly consistent with those of previous observational studies.^25,27–30 Evidence from a comprehensive systematic review and meta-analysis of 11 randomised control trials involving 161,388 participants and 46 observational studies involving 55,367,053 participants showed that 11 COVID-19 vaccines were effective against five SARS-CoV-2 variants of concern: Alpha, Beta, Gamma, Delta and Omicron.²⁷ This systematic review found that in the primary COVID-19 vaccine series, the summary measure of overall VE was 88.0% against the Alpha variant, 77.8% against the Delta variant, 73.0% against the Beta variant, 63.0% against the Gamma variant and 55.9% against the Omicron variant. Against the Delta variant, the VE of the booster vaccination was 95.5%, and against the Omicron variant, the summary VE of the booster vaccination was 80.8%.

In a recent systematic review and meta-analysis involving 68 studies from more than 23 countries, VE for the primary COVID-19 vaccine series at baseline (14–42 days) was 92% for hospitalisations and 91% for mortality. This effectiveness was reduced to 79% at 224–251 days for hospitalisations and 86% at 168–195 days for mortality.²⁸ Against all documented infections, VE was 83% at baseline (14–42 days), decreased to 62% by 112–139 days and then decreased gradually to 47% by 280–307 days. At baseline, the booster doses of the COVID-19 vaccines showed 70% effectiveness in preventing infections and 89% effectiveness in preventing hospitalisations. However, this effectiveness decreased to 43% against infections and 71% against hospitalisations after ≥112 days.

Similarly, a large observational study conducted by Lin et al. described the trajectory of the waning effectiveness of the COVID-19 vaccine (double dose regiment) over 9 months (11 December 2020–8 September 2021) in North Carolina, United States of America.²⁵ The three vaccines assessed were BNT162b2, mRNA-1273 and Ad26.COV2.S. The VE against COVID-19 infections was 74.8–85.5% in the 1st month, increased to 71.4–95.9% in the 2nd month, then waned gradually to 67.8–77.8% by the 8th month. Against hospitalisations, the VE was 85.8–96.4% in the 2nd month and decreased to 81.7–94.3 by the 6th month. Against mortality, VE ranged from 65.5% to 91.6% in the 1st month, increased to 82.2% to 98.6% in the 2nd month, and gradually decreased to 71.2% to 92.5% during the 6th month.

A strength of our study is that we conducted a 1:1 matched retrospective cohort study using nation-wide individual-level data from the Ministry of Health and the ESR, improving the accuracy and completeness of vaccination status, COVID-19 cases, hospitalisation and mortality data. Additionally, we controlled for demographic factors, comorbidity and multimorbidity, susceptibility, infectiousness and immunity (vaccine induced, infection induced and hybrid immunity). This allowed us to control for confounding and enabled us to examine the trajectory of COVID-19 VE as a function of time since vaccination. We have previously used this approach to assess the effectiveness of shingles vaccines in Aotearoa New Zealand.⁸

Our study weaknesses include vaccine coverage being very high in Aotearoa New Zealand, with VE as a function of time post-vaccination in the sub-group analysis becoming under-powered because of the dwindling numbers of eligible unvaccinated populations. Also, under-reporting and unmeasured confounding, particularly in the study when the unvaccinated population was small, may have resulted in an under-estimation or over-estimation of the VE point estimates.^31,32 Potential confounders that are not included in our model but could increase point estimates for VE include personal behaviours (i.e., if the vaccinated had higher mask use and greater social distancing) and antivirals (made available for at-risk populations from May 2022).

Although multiple vaccines are approved and available in Aotearoa New Zealand, almost all people received Pfizer–BioNTech (Comirnaty). However, we could not access the data required to perform a sub-group analysis by vaccine type. Also, VE against specific variants could not be assessed as the Omicron variant predominated during the study period.

RATs were included in our estimates for VE against infection (after 16 February 2022). RATs were used by the public to enable self-management and their use was therefore subject to individual testing behaviours (rather than PCR tests used systematically in clinical settings e.g., to influence treatment options). For instance, individuals may have tested with RATs more frequently if they lived in households with individuals at greater risk of severity (e.g., older adults and/or persons with compromised immune systems).³³ Vaccinated individuals were also twice as likely as unvaccinated individuals to express their intention to undergo COVID-19 testing and report being tested in the past month.³³

Our findings suggest that COVID-19 vaccines provided longer-term protection against hospitalisation and mortality and shorter-term protection against infection. VE decreased gradually as a function of time post-vaccination, especially against SARS-CoV-2 infection. These findings are based on the use of historical time frames (i.e., up to 28 February 2023). This may lead to discrepancies when compared to alternative analyses or official data (e.g., that which encompass the most up-to-date COVID-19 data).

Future observational studies using well-powered national data will be needed to assess the effectiveness of COVID-19 vaccines beyond 6 months post-vaccination. It is vital to evaluate VE by vaccine type and variant. Additionally, there is a need to understand how VE differs for immunosuppressed people at different stages of disease and treatment. Policymakers, clinicians and patients will be reassured by the VE estimates found for the COVID-19 vaccine, particularly against severe outcomes (hospitalisation and death). Given the waning effectiveness of these vaccines over time, however, further surveillance should be undertaken to monitor vaccine uptake among groups and the VE of future booster doses (including for bivalent vaccines) or combination vaccines. Also, there is a need to evaluate the cost effectiveness of COVID-19 vaccinations and booster doses specifically in Aotearoa New Zealand. Economic evaluation evidence could help guide decisions on routine annual vaccination coverage, particularly in situations with limited vaccine supply. This information could also inform whether the government should prioritise alternative strategies, like improving ventilation in public settings.

View Figure 1–2, Table 1–5.

View Appendix.