A new national health system: the opportunity to address data quality issues in maternal immunisation coverage

Maternal immunisation against pertussis and influenza is criticalto prevent hospitalisation and potentially fatal outcomes during pregnancy and inearly infancy.1 While maternal immunisation for both pertussis andinfluenza in Aotearoa New Zealand has increased since 2013, it remains suboptimaland inequitable,1,2 and obtaining accurate data on maternal immunisationis fraught with challenges. The recent New Zealand Health and Disability SystemReview3 focused on the need for system-wide approaches to ensurethe health system achieves better and equitable outcomes. The current environmentof health reform presents a timely opportunity to address the challenge of low maternalimmunisation coverage, which requires high quality data on immunisation coverage.4,5 (sup: TM)

In Aotearoa New Zealand, vaccination in pregnancy has been government funded through general practice (GP) and hospitals nationwide since 2010 for influenza, and 2013 for pertussis. Delivery through pharmacies has been funded since March 2017 for influenza and September 2022 for pertussis. There is an annual drive to immunise the population against influenza which includes extensive advertising and media coverage, as well as a concerted government-funded effort to vaccinate high-risk patients, including those who are pregnant. There has been less attention to pertussis vaccination in pregnancy. In previous work examining maternal coverage in Aotearoa New Zealand, seasonal variation in coverage for influenza was identified,2with a peak at the start of the influenza season, declining later in the year.

Capturing clear maternal immunisation coverage in Aotearoa New Zealand is not straightforward due to the number of providers, the funding arrangements, and how they both claim and record vaccination events. Even determining pregnancy status at the time of vaccination can be problematic due to the status being unknown or not discussed by the provider.

In Aotearoa New Zealand, an immunisation event can be captured using multiple data sources: the National Immunisation Register (NIR), Proclaims, and the Pharmaceutical Collection. The NIR began as a register for the Meningococcal B (MeNZB) vaccination campaign and, from 2006, evolved into a register for all childhood immunisation enrolments and events, as per the New Zealand National Immunisation Schedule. From 2013, the NIR increasingly captured selected adult scheduled immunisations, including those given during pregnancy (pertussis and influenza). The Proclaims and Pharmaceutical Collection datasets contain data on the fee-for-service payments made to GPs or community pharmacies, respectively, for providing government-funded immunisations. Workplace influenza vaccinations have not been well captured by either system; it is recommended to notify GPs of an influenza vaccine receipt, but this relies on both the vaccinee informing the workplace service of the correct GP and that completed vaccinations are communicated accurately to the practice and uploaded onto the NIR. It is unknown how effective this is, but it is certainly not complete.

The objective of this study was to quantify discrepancies in maternal immunisation coverage using the Te Whatu Ora – Health New Zealand QLIK data platform and a range of data sources, including NIR, Proclaims, or Pharmaceutical collection.

Methods

Consistent with previous publications,^1,2 we determined maternal coverage for influenza and pertussis using the Aotearoa New Zealand administrative health data sources in the following way: the study population (denominator) consisted of all pregnant people with a delivery between 1 January 2013 and 30 June 2021 in the Maternity collection dataset. People were excluded (n=32,063) if the gestational age at delivery was less than 20 weeks or greater than 45 weeks, if either the date of last menstrual period or gestational age at delivery was missing, if maternal age at delivery was less than 12 or greater than 50 years of age, and if flagged as a non-resident. The numerator was receipt of an influenza or pertussis vaccine during pregnancy, determined by a valid entry for a pertussis and/or influenza vaccine in available data sources (NIR, Proclaims, or Pharmaceutical collection) during their eligible pregnancy within the cohort. Available sources of immunisation information were prioritised in the following order: NIR, Proclaims, then Pharmaceutical Collection and an immunisation was considered valid if it occurred between the last menstrual period and delivery date, as recorded in the Maternity collection. Immunisation data was linked to the Maternity collection via an encrypted National Health Index identifier. We determined the proportion of immunisation records captured outside the NIR in claims data (Proclaims or Pharmaceutical collection) and, additionally, compared our study data to the coverage data provided by Te Whatu Ora – Health New Zealand via their QLIK platform. All statistical analyses were undertaken using SAS Enterprise Guide (9.4) statistical software (SAS Institute Inc., Cary, NC, USA). This study was approved by The University of Auckland Human Participants Ethics Committee (Ref. 022536).

Results

Our findings show that since 2014, the number of maternal influenza immunisations identified by the NIR data has grown from less than 55% of all immunisations to just over 90% (see Figure 1, Panel 1A). However, in 2021, almost 10% of maternal influenza immunisations were only identified by using the GP claims or pharmaceutical claims databases, with similar findings for maternal pertussis immunisations (see Figure 1, Panel 2A). Pertussis did not become funded in pharmacies until 2022, so there was not any pharmaceutical claims information until after this date. A large discrepancy was identified between our study data coverage to that provided by Te Whatu Ora – Health New Zealand (see Figure 1, Panel 1C). The pertussis data provided by Te Whatu Ora is similar to our datasets, with a slightly higher coverage in QLIK data from 2019 onwards. There were large differences between the influenza datasets, showing that there is a large underestimate in the Te Whatu Ora data in maternal influenza immunisations. It is important to note that the significant drop in 2021 influenza study data is due to incomplete data.

View Figures 1A-1C.

Discussion

Our study aimed to quantify discrepancies in maternal immunisation coverage using the Te Whatu Ora – Health New Zealand QLIK data platform and a range of data sources, including NIR, Proclaims, or Pharmaceutical collection. Our findings clearly demonstrate that while an increasing number of maternal immunisations are being captured in the NIR, there remains a data discrepancy of up to 10% between QLIK reporting and the combined NIR and claims datasets. There is currently no gold-standard dataset for reporting immunisation coverage in Aotearoa New Zealand; however, if researchers or policymakers are aiming to determine maternal immunisation coverage they should utilise as many of these different sources as possible, as has been the case for recent publications.^1,2 Understanding why immunisation events are not being captured is critical if we are to improve the way in which data is collected and then utilised. For instance, it is unclear why coverage is higher in QLIK data than our study data for pertussis, but lower for influenza. This could be due to events not being entered into the NIR or data entry into the NIR not being coded for pregnancy. There are several plausible systems reasons for these errors. Considering where these gaps are occurring will be important as the new AIR is being designed.

Te Whatu Ora – Health New Zealand provides NIR data via their QLIK platform to their approved users to better support service delivery and to improve immunisation coverage, including summary statistics and trends on maternal influenza and pertussis coverage.⁶ The same trend is seen between the QLIK NIR and our study coverage data. However, QLIK estimates similar coverage for pertussis and much lower coverage for influenza compared to our study data, which utilised multiple data sources. This is despite, in theory, using the same source data for the denominator (the maternity collection). The degree of data cleaning and exclusion and inclusion criteria Te Whatu Ora – Health New Zealand is using to determine the denominator and how they determine administration of a vaccine during pregnancy from the NIR is unclear, and not well defined for replication or allowing for understanding of bias in the numerator or denominator. It is also worth noting that the claims based administrative datasets (GP and pharmaceutical claims, and the maternity collection) have a lag of up to 12 months, reducing the ability of timely analysis and reporting of maternal coverage. To some degree Te Whatu Ora – Health New Zealand gets around this limitation in their QLIK data by only using the NIR as the numerator (lags 1–3 months) and using provisional or previous year maternity data (denominator), but at the expense of accuracy.

Maternal immunisation, the first vaccine event in the life-course immunisation programme for a child, remains important to prevent influenza and pertussis-related adverse outcomes, as well as an important opportunity to engage with whānau around immunisation. Capturing maternal immunisation coverage in Aotearoa New Zealand is currently problematic and several data publications, including Te Whatu Ora – Health New Zealand, are likely to be underestimating coverage. We have shown that relying on the NIR as a single data source is not a reliable option, and neither is relying solely on funding claims. To be entitled to free maternal immunisations, individuals must disclose their pregnancy, which enables the correct coding of pregnancy with the vaccination event into the NIR. However, there are many reasons pregnancy may not be disclosed, e.g., for some, the cost of the vaccine is not a large barrier compared to convenience and discretion (if wanted in early pregnancy), so there is no obligation to disclose pregnancy to receive a vaccine. It has been shown that administrative health data that relies on accuracy from claiming for funding is notoriously inaccurate and surveillance systems that capture immunisation coverage accurately assists with increasing coverage.^7,8 With no gold-standard, it is unclear how inaccurate the NIR may be; however, some estimates suggest it could be up to 10% from true immunisation coverage.⁹ In addition, the NIR is known to be inaccurate for children’s coverage when compared to data from GP practice management systems or the Well Child book.^9,10 Therefore, it is unlikely that we are going to get a reliable full coding of “pregnancy” status in the immunisation register going forward without both changes to incentives to improve a focus on pregnancy, alongside greater attention to how data is entered at the vaccinator level to minimise the risk of missing pregnancy as a code. Ultimately, reliance on the NIR alone under-reports vaccination coverage in pregnancy. The change to Aotearoa New Zealand’s health system presents an opportunity for a nationally coherent strategy around collection and presentation of important health statistics, especially if health targets are ever considered again. These health system changes also allow for important conversations around delivery, incentives, equity, and governance of immunisation, as well as data sovereignty and the use of overseas data storage.¹¹ Users of Aotearoa New Zealand administration health data and statistics require confidence in their determination and thus interpretation.

Conclusion

Increasing numbers of maternal influenza and pertussis-containing immunisations are being captured in the NIR. However, around 10% continue to remain outside the NIR, leading to inaccuracies in reporting. While there is currently no gold standard, the improved capture of maternal immunisation data is needed to ensure accurate reporting and monitoring of immunisation coverage.