Accuracy of ethnicity records at primary and secondary healthcare services in Waikato region, Aotearoa New Zealand

In Aotearoa New Zealand, ethnicity is defined as the “ethnic group or groups that people identify with or feel they belong to”.¹ Ethnicity is a social construct, self-perceived, can change over one’s lifetime and recognises that people can identify with more than one ethnic group.¹ In Aotearoa New Zealand healthcare, accurate ethnicity data collection is essential for public health prioritisation, policy planning/making, monitoring, eligibility for services and resource allocation. Inaccurate data collection of ethnicity impacts these processes, effects access and outcomes for populations and adversely impacts the Crown’s obligation to Te Tiriti o Waitangi to achieve health equity, as well as Māori rights to monitor the Crown.² The 2017 updated Ethnicity Data Protocols released by the then Ministry of Health³ aimed to build on the earlier 2004 iteration⁴ to standardise the collection, recording and output of ethnicity data within the Health and Disability sector, in-line with the Statistics NZ statistical standard for ethnicity that applies to the Census and across government agencies.⁵ Despite these protocols being in place since 2004 in the Aotearoa New Zealand public healthcare system, multiple subsequent groups have reported high levels of inaccuracy and undercounting of Māori.^6–10

This research used prospectively in-person collected self-reported ethnicity data, gathered as part of the “Te Whakangungu Rākau” study¹¹ investigating thyrotoxicosis, to audit the accuracy of ethnicity recording in primary care and national hospital accessed datasets. As an issue central to Indigenous rights, this analysis focusses on the accuracy of ethnicity data for Māori.

Method

Ethical approval was obtained from the National Health and Disability Ethics Committee (HDEC), Waikato DHB and Te Puna Oranga prior to commencing the study (13/NTB/4).

Self-identified ethnicity data were gathered from 475 participants between March 2013 and December 2014 as part of the “Te Whakangungu Rākau” (WNR) studies (353 participants from prospective incidence cohort study¹¹ and 122 additional participants from a prospective radioactive iodine cohort). Eligible participants were identified from patients referred to a public or private specialist endocrine service in the Waikato region of Aotearoa New Zealand. The WNR study collected in-person ethnicity data from participants using a standardised written questionnaire paralleling the Aotearoa New Zealand Census question⁵ and as recommended by Ministry of Health – Manatū Hauora ethnicity protocol.³ Multiple responses were allowed for (as many as required), and results were recorded at the most detailed/disaggregated level 4 ethnicity classification.³ This ethnicity is considered the gold standard for this study and is referred to as “self-reported WNR ethnicity” from here on.

For all participants who consented for their information to be used in future studies (one participant excluded, n=474), their self-reported WNR ethnicity was compared against two different datasets at primary and secondary healthcare levels, with data extracted over the summer of 2018–2019. Primary care ethnicity data was requested from Hauraki Primary Health Organisation (n=157) and Pinnacle Primary Health Organisation (n=213) and from a handful of single general practices (GPs) (n=10) for patients registered with these primary care services. The remaining patients (n=94) were not registered with a GP, or there was no record of a registered GP. From 2016, primary care practices have been able to link their practice management systems with the National Health Index (NHI), enabling them to access and update the ethnicity recorded in the NHI.¹² Secondary healthcare ethnicity data was extracted from the Waikato District Health Board i.Patient Manager (iPM), which was able to draw live from the NHI dataset. This dataset is referred to hereafter in this manuscript as the “secondary NHI” dataset.

Excluding ethnicity, all other demographic data was taken from the WNR studies. Age was calculated from the date of birth to 1 January 2019, when this project’s new data was extracted. Experience of material deprivation was collected prospectively during the WNR studies (therefore 2013–2014), using the New Zealand Index of Deprivation (NZDep) eight-point individual questionnaire.¹³

Ethnicity within each dataset was extracted at the most granular level available (level 4 for self-reported and level 2 for national, secondary and primary datasets). When necessary, this was coded to appropriate level 1 ethnicity groupings as per protocol.³ Ethnicity was categorised and compared as total ethnicity, and to manage multiple ethnicity both prioritised ethnicity and sole/combination (nine possible categories: European, Māori, Pacific peoples, Asian, Other, Māori/European, Māori/Pacific peoples, “Two groups Not Elsewhere Identified” or “Three groups”) as per protocol.³ Both different datasets (primary and secondary) were compared to the self-reported ethnicity data to assess the accuracy of ethnicity within these commonly used datasets. For prioritised and sole/combination ethnicity outputs, these were considered either concordant (i.e., the same) or discordant (i.e., not exactly the same.) Given multiple ethnicity potential within total ethnicity output, congruence was considered concordant (all recorded ethnicity/ethnicities the same), partially concordant (at least one, but not all ethnicity/ethnicities the same), or discordant (none of the ethnicity/ethnicities the same). Participant demographics (age, gender and NZDep) were used to investigate factors associated with congruence. As the primary care cohort was incomplete (n=380), they were compared to the self-reported WNR ethnicity responses of the same individuals.

When aggregating ethnicity to level 1, some individuals with multiple ethnicity responses became allocated to single ethnicity response categories (e.g., at level 2 one individual is Niuean and Tongan, which becomes one Pacific peoples ethnicity). This did not happen in the self-reported WNR cohort but occurred four times in the secondary NHI dataset (three European, one Pacific peoples), and three times in the primary care (all European). Where the level 1 ethnicity matched the gold standard self-reported WNR ethnicity, this was considered congruent (even though at the lowest level these where not actually congruent). The eight “not stated” responses for second ethnicity in the primary care dataset were not counted as multiple ethnicities, as this was considered a void statement, while “response unidentifiable” were.

Statistical analysis was performed on Stata/SE 16 (StataCorp. 2019. College Station, TX: StataCorp LLC) using Chi-squared tests for count variables to compare congruence between dataset and Mann–Whitney for non-parametric variables. A logistic regression analysis of prioritised ethnicity concordance compared to discordance was used to determine if demographic variables such as age, ethnicity and deprivation are associated with any inaccuracies seen.

Results

The cohort consisted of 474 participants, of which 390 (82.3%) were females, the median age was 55 years (range 19 years to 98 years) and using the NZDep, 29.1% of individuals lived with two or more measures of material hardship (8.6% lived with five or more). Collectively, the 474 individuals had 515 self-reported WNR ethnicities, with the total response ethnicity at each of the four ethnicity levels presented in Table 1.

Secondary NHI ethnicity records were available for all 474 participants. Primary care ethnicity records were available for all participants (n=380) requested. The level 1 total response ethnicity from the three datasets are shown below in Table 2. Total response ethnicity at level 2 is available in Appendix Table 1.

There was a net difference of 6.1% fewer Māori recorded in the national secondary NHI dataset and 5.6% fewer in the primary care dataset when compared to self-reported WNR ethnicity (secondary NHI 10 fewer and primary care 7 fewer individuals) (Table 2). Pacific peoples were also under-represented at level 1, although numbers are small. Table 3 presents the overall accuracy of records, with some discrepancy in record noted in 9.5% of secondary NHI records and 11.1% of individuals in primary care records when compared to their self-reported WNR ethnicities.

Multiple ethnicity

At the most disaggregated available level, 60 individuals (12%) had more than one ethnicity documented in at least one of the datasets. Māori were more likely to report multiple ethnicity (23.9% of Māori reported multiple ethnicity). The self-reported WNR cohort recorded higher multiple ethnicity (8.7%) compared to secondary NHI dataset (5.3%), and primary care dataset (5.8%) (Table 2). Māori and European (level 1) ethnic grouping was the most common combination (self-reported in 6.5% of the WNR cohort).

Table 3 illustrates discordance of ethnicity records compared to self-reported ethnicity when multiple ethnicities are managed by categorisation using total response, prioritisation or sole/combination. For all three datasets, prioritised ethnicity had the highest amount of concordance with self-reported WNR ethnicities, ranging between 93.7–95.1% concordance. Total response and sole/combination (9 options) had similar concordance (88.9–90.5%), but had different ways of managing the discordance.

Factors influencing ethnicity discordance

Individuals who self-reported more than one ethnicity (41 individuals) had more discordance in the datasets than those who reported a single ethnicity (Figure 1). The concordance for the group of individuals with multiple ethnicity was improved from ~36% full concordance with either total ethnicity or single/combination ethnicity to 70.7–75.8% with prioritised response (Figure 1), as the most common discordance was the exclusion of one ethnicity in an individual with multiple ethnicities.

Figure 2 demonstrates differences in concordance between the two datasets when compared to the self-reported WNR prioritised level 1 ethnicity of Māori, European and a conglomerate of all other ethnicity options (“Other”). Māori had much lower amounts of concordance (~78%), similar to the Other group (77–84%), while European individuals had records with 97–99% concordance (p<0.005 for both the secondary NHI records and the primary care records). For Māori, when self-reported WNR ethnicity was compared to secondary NHI records, 31 individuals were partly congruent (24 individuals with multiple ethnicities not having one of their ethnicities recorded, one individual with multiple ethnicities having a different ethnicity recorded and six individual’s single ethnicity having an additional ethnicity recorded) and three were discordant (two recorded as European and one residual categories). For non-Māori, three were partly concordant and eight discordant (two residual categories were recorded as European and one European was recorded as residual categories, two Pacific peoples were recorded Māori, one Asian as Pacific peoples, one Asian as Māori and MELAA as European).

Comparing self-reported WNR total ethnicity to the secondary NHI total ethnicity, the discordant cohort was more likely to have identified as Māori in the self-reported study (70.7% compared with 29.3%, p<0.001) and be of a younger age (median age 45.1 years [IQR 21.4] compared with 50.1 years [24.3], p=0.035), but the relationship to gender (p=0.062) or deprivation (p=0.069) was not clearly explained. Logistic regression comparing self-reported total ethnicity to national records total ethnicity (Table 4) demonstrates Māori ethnicity and reporting multiple ethnicities were the only factors independently associated with discordance of ethnicity.

Discussion

The accuracy of ethnicity in administrative datasets has been and continues to be an issue in Aotearoa New Zealand, particularly for Māori and other non-European ethnic groups. In 2022, Harris et al. found that when individually linked ethnicity data were compared to the Census ethnicity, the NHI under-counted 16% of Māori. Swan et al. reported in 2006 that only 72.2% of Māori were correctly recorded in hospital records compared to 99.3% of non-Māori.⁸ Riddell, in 2008, also shows that primary care records were correct for only 64.9% Māori and 90.9% for New Zealand Europeans.⁷ Rumball, in 2011, showed the same inaccuracies, as Māori had only 71.2% of their ethnic groups recorded accurately and non-Māori had 99.3% accurately recorded.⁹ Meanwhile in 2018, and also within the Waikato District Health board, accuracy was recorded at 79.3–82.8% for Māori patients who presented with a traumatic injury.⁶ In the current study, the discordance rates were similar to these historical reviews, with accuracy ranging from 75–91.5% for Māori, with discordance also seen in other minority ethnic groups, and Māori ethnicity (self-reported, prioritised) was associated with a 0.36 odds ratio (p=0.039) of having ethnicity concordance in the secondary NHI dataset. Despite strict and clear policy on the collection of ethnicity data within the healthcare system, an unacceptable amount of inaccuracy still remains.

Our study shows that when gold-standard ethnicity collection is used, people report more ethnicities than are currently recorded in health datasets. This indicates our current health data systems are failing to fully capture ethnic affiliations, especially for Māori, who are more likely to report multiple ethnicities. In addition to Māori or other non-European ethnicities, identifying with more than one ethnicity was strongly associated with likelihood of discordant ethnicity health records (odds ratio 0.05, p<0.001). Ethnicity records must be able to accurately collect and document multiple ethnicity options. In this cohort, 8.6% of those in the self-reported WNR data identified with more than one ethnicity. Twenty-four percent of the Māori WNR cohort identified with more than one ethnicity. Despite the Ministry of Health Ethnicity Protocols³ explicitly stating that ethnicity should be collected and stored at level 4 disaggregation, and with up to six potential ethnicities per individual, both primary care and secondary NHI datasets had only level 2 specificity data, and only three potential ethnicities per individual. In the Census and other key datasets, reporting of multiple ethnicities is common, especially for Māori and younger peoples, so is an issue that impacts these groups more if this is not being captured accurately. As such, the healthcare datasets need to be able to appropriately represent this population as accurately as it does those with a single ethnicity.

Our study demonstrates a concerning level of discrepancy between self-reported ethnicity and that recorded in administrative datasets, with 9.5–11.1% of individuals having at least some inaccuracy in their ethnicity recorded in health datasets when compared to self-reported WNR data. While the net degree of inaccuracy was similar across both datasets, it is important to note that they were not in the same individuals. This finding indicates that all administrative health datasets continue to have quality issues, and that data integration through the common NHI record is not yet in practice.

Appropriate reporting of multiple ethnicity entries continues to be a matter of discussion in Aotearoa New Zealand.¹⁴ Total ethnicity continues to be the preferred manner of ethnicity reporting in research and population data. In this cohort, total ethnicity reporting of secondary NHI ethnicity datasets led to a net 5.6–6.1% undercount of Māori. It is not clear how much of this is due to the undercounting of Māori in multiple ethnicities and how much of this is due to people who only identify as Māori being misclassified as another ethnicity. This needs to be considered when presenting ethnicity information from these datasets. The use of prioritisation, as a system to manage multiple ethnicities, had higher overall concordance (93.7–95.1%) and Māori ethnicity concordance (89.0–91.5%).

These discrepancies in the quality of the healthcare datasets in Aotearoa New Zealand have significant implications for public health prioritisation, policy planning/making, monitoring and resource allocation and are a breach of the Crown’s obligation to Te Tiriti o Waitangi to achieve health equity. It is impossible to measure the impact of policies that are focussed on health inequities between different ethnicities without accurate and high-quality ethnicity data. A timely reminder of this was the COVID-19 pandemic and the healthcare response to it. Many measures that were monitored to document the national experience of COVID-19 used national ethnicity data, e.g., PCR testing, positive cases, hospitalisations and deaths. The national roll out of the COVID-19 vaccination also relied on national healthcare ethnicity records (e.g., NHI) to monitor the equity of vaccination coverage in Māori and Pacific peoples communities, who were priority target groups. Inaccuracies in these datasets will likely cause critical flaws in this approach, resulting in inaccurate coverage data and Māori individuals missing out on targeted services.

This study adds further evidence of inaccuracies in ethnicity data recording, and discrepancies in ethnicity between Aotearoa New Zealand key health datasets. This provides information relevant to addressing numerator-denominator bias i.e., demonstrating an undercount for Māori and poor collection of multiple ethnicities in health data, with implications for the use of different types of outputs. As each of the data sources assessed in this study are used for research and policy, the ability to compare across them is a strength of this paper. In addition, this is one of the first projects to consider the impact of different approaches to multiple ethnicities on the accuracy of ethnicity within a cohort. The fact that the original cohort was part of a Kaupapa Māori research project that prioritised the importance of excellent quality ethnicity data is also a strength.

The discrepancies found in our study are above and beyond what could be explained by other factors. Ethnicity mobility has been used to describe how ethnicity can change over time. There was a 5-year time difference between the collection of the self-reported WNR ethnicity and the primary and secondary NHI dataset extraction. As such, ethnicity may have changed for some individuals. Evidence shows that this mobility is generally low in Aotearoa New Zealand, with just over 5% of Māori individuals changing ethnicity between each Census, with minimal net change to the Māori population numbers at an aggregate level.¹⁵ Ethnicity responses may differ due to a range of factors, such as the environment the question is asked, question design and the perceived benefit or consequence that may come with the question.^3,15 It is therefore possible that the Kaupapa Māori research environment of the self-reported WNR ethnicity led to some variation in expression compared to healthcare interactions. However, the ethnicity record from secondary NHI and primary care records were taken from a single time-period, so mobility may be even less a factor. The inconsistencies between these datasets raises quality concerns, especially as the individuals with discordant ethnicity were not the same across both databases. Further investigation is needed to determine whether this is due to deviating health service approaches to ethnicity data collection, or differences in the perceived safety of reporting Māori ethnicity in healthcare settings. Moreover, the NHI database is now a centralised computerised record, which all levels of the health system should be accessing and updating. So, a discrepancy between primary care and NHI ethnicity indicates that these systems are not fully integrated in practice. The ethnicity protocols stipulate that ethnicity should be asked every 3 years, and that at each healthcare interaction (whether in hospitals or primary care) there is the opportunity to update the patient’s self-reported ethnicity in the NHI record. More work is needed to improve ethnicity data recoding in every healthcare setting.

This paper examines ethnicity data quality by investigating differences in prioritised, total and sole/combination ethnicity outputs from primary and secondary care data compared to gold-standard collection of ethnicity in the WNR study. However, it does not suggest a preferred option for the analysis of ethnicity data. This will depend on the purpose of the research and an understanding of the strengths and limitations of the datasets, including regarding ethnicity data.¹⁶

The study participants themselves were not representative of the total population, being drawn from a cohort of all local adults (>15 years of age) presenting with a first diagnosis of thyrotoxicosis between January 2013 and October 2014. As a result, the participants were predominantly (>80%) female—ethnicity inaccuracy¹⁰ and mobility¹⁵ have been found to be markedly higher for Māori males, so our study is likely to underestimate the inaccuracy of ethnicity data for the total Māori population. This study also samples only individuals who are health service users, and thus may not be representative of the total population, especially for Māori, who are less likely than non-Māori to receive health services.¹⁷

Nevertheless, our study shows that ethnic identity is recorded less accurately for Māori, for other non-European individuals and for those who have multiple ethnicities. Self-reported ethnicity collection is paramount to achieving correct ethnicity reporting. More effort is needed to improve ethnicity data collection, in particular for those who identify as having multiple ethnicities recording and reporting to improve the accuracy of counting Māori in health datasets. Specifically, health services at all levels must implement the existing national ethnicity protocols, and compliance should be a core focus for regular self-audit, as well as a requirement for accreditation and funding. Staff training in ethnicity data collection and cultural safety is needed to ensure it is safe for Māori to report ethnicity data in all healthcare settings, and data are collected (and updated) accurately. Software barriers to the appropriate recording of multiple ethnicities in health datasets must be urgently addressed. The findings of this study further underscore the urgent need to implement the actions called for by Te Aka Whai Ora – Māori Health Authority, in its Action plan for achieving high quality ethnicity data in the health and disability sector released in 2023.¹⁸

 View Table 1–4, Figure 1–2.

View Appendix.