Our health (data) is our wealth: improving primary healthcare data access for research in Aotearoa New Zealand

Routinely collected primary healthcare data can be used to conduct cost-effective research that provides real-world health insights and health benefits. Such research has wide-ranging potential to impact health, economic and socio-cultural outcomes and health policy.¹ Health researchers can use primary healthcare data to identify which treatments work best for specific patient groups.² Health outcomes can be improved through better understanding of disease, leading to more effective prevention strategies, earlier diagnosis and better-informed models of health. Research provides the necessary evidence to support system-level health policy change. This has the potential to increase equity, as well as the alignment and influence of health priority goals. Within an Aotearoa New Zealand context this is especially important because of the impact that colonisation and racism have on health outcomes for Māori (Indigenous New Zealanders).³ For multiple medical conditions, Māori are less likely to receive timely diagnosis and treatment compared to Pākehā (New Zealand Europeans).³ Any research that occurs through New Zealand primary healthcare data analysis needs to benefit all New Zealanders. Health improvements resulting from the above can contribute to broader wellbeing, social participation and workforce productivity.

Primary healthcare is the entry point to the New Zealand health system, with around 75% of the population visiting a general practice annually. All clinical and administrative activities undertaken in primary healthcare settings are recorded in the electronic health record or patient management system (PMS), for example clinician notes, diagnoses, prescriptions and test results. These data contribute to, and can be linked with, various national datasets for research purposes.⁴ Primary healthcare records contain granular data across the human lifespan: given the widespread early use of PMS in New Zealand, up to several decades of individual patient-level data are potentially available, including whānau (family) history of disease, past or current acute and chronic disease, abnormal clinical events or measurements and procedures, and prescribing and other summary data.

Primary healthcare data offers considerable potential for understanding the aetiology and sequelae of disease. However, there are some major caveats to using these data; it is often stored in free-text form or idiosyncratically coded, potentially limiting its utility for research.^4,5 Accessing national-level data is difficult because there is no single primary healthcare data repository in New Zealand. Across the health system, health data are separated by discipline and care setting. Primary healthcare data are further siloed by various point-of-care PMSs operating in individual practices. Primary healthcare data are not well-linked to other datasets. Collation of some health data occurs via primary health organisations (PHOs) contracted by the government to provide regional primary healthcare services, but this is principally for funding and service allocation rather than research. National health datasets are relied upon for research and policy development, such as the National Minimum Data Set (births, deaths, hospital admissions and discharges) and the Pharmaceutical Collection (prescriptions). However, these data provide only an incomplete understanding of population health and service use. The largest New Zealand research dataset, the Integrated Data Infrastructure (IDI), includes general practice enrolment and community prescribing data, but no other primary healthcare data (see: https://www.stats.govt.nz/integrated-data/integrated-data-infrastructure/).

Primary healthcare research networks

One solution to improve access is to develop a research network which provides the infrastructure to collate and link health data for research purposes.^6,7 This concept underpins the regional Southern Primary Care Research Network (SPCRN) (Otago and Southland) and is used across the South Island by Te Waipounamu data collective.^4,8 SPCRN data access is controlled by a governance group, co-governed 50:50 with Māori; members are clinicians, Pacific peoples, community and academics. Data access is limited to accepted researchers who have gained ethical approval for a specific project.⁴

The research network approach has been successfully used for decades internationally.⁶ The United Kingdom (UK) makes particularly good use of primary healthcare data via numerous research networks, including the Clinical Practice Research Datalink (see: https://www.cprd.com), QResearch (see: https://www.qresearch.org) and the Oxford-RCGP Research and Surveillance Centre (see: https://www.rcgp.org.uk). The United States of America (USA) also uses these routinely collected data effectively via practice-based research networks, supported by the Agency for Healthcare Research and Quality (see: https://www.ahrq.gov/ncepcr/communities/pbrn/index.html), as do many other countries.⁶

Technical approaches to primary healthcare data access

To overcome the current challenges of accessing primary healthcare data in New Zealand, three new complementary methodological and technical solution pathways are described here as 1) platform-level, 2) algorithm-level and 3) data-level approaches, together with the advantages and limitations of each (Box 1).

View Box 1.

1. Platform-level approaches provide a secure, auditable infrastructure which allows accredited researchers to access de-identified data within controlled settings, as exemplified by the Trusted Research Environment (TRE) in the UK.^2,9 TREs can support integration of primary healthcare data with other national datasets, such as hospitalisations, pharmaceutical dispensing records or other clinical or research data. This integration greatly facilitates comprehensive health surveillance and research endeavors. Approved researchers can write and execute analysis code within the TRE, avoiding the need for data extraction and transfer. Outputs leave the TRE in the form of aggregated non-disclosive statistical results. However, TREs require significant financial and technical investment and may need to rely on offshore cloud providers, raising data sovereignty concerns.¹⁰ The New Zealand context requires that development of a similar TRE would need to be co-designed by government, Māori stakeholders and other interested parties. Governance structures would need to include Te Tiriti o Waitangi, Māori data sovereignty and Māori data governance to facilitate community trust and ethical stewardship.^11,12
2. Algorithm-level approaches include federated learning, which is a decentralised machine learning technique that trains models collaboratively across distributed data holders without transferring raw data outside its source.¹³ Within this system, only encrypted model parameters are shared and aggregated, preserving patient privacy. Federated learning supports diverse modelling frameworks, from complex deep neural networks to interpretable statistical models, enhancing health research applicability. However, federated learning is based on an original dataset and therefore carries the risk of perpetuating bias and racism contained within that dataset. Also, heterogeneity in coding standards, data structure and variable patient record quality challenges model fairness, reliability and generalisability. Overcoming these challenges requires investment in harmonised data standards, robust infrastructure and rigorous validation protocols.¹⁴
3. Data-level approaches focus on synthetic data generation, creating artificial datasets that are statistically representative of real-world data. A synthetic dataset mimics but does not directly replicate patient data, protecting confidentiality.^14,15 Synthetic data generation is considered best practice internationally for accessing sensitive health data. The UK has developed a synthetic primary healthcare dataset, OpenSAFELY (see: https://www.opensafely.org). The original data remain in situ within the general practices, and no data transfer occurs (see: https://digital.nhs.uk/services/artificial-data). Recent advances in generative artificial intelligence (AI) have expanded the methods available for synthetic health data generation. Early models like generative adversarial networks (GANs) and variational autoencoders (VAEs) laid the groundwork for learning complex data patterns; newer approaches such as diffusion models and large language models (LLMs) have significantly improved the realism, diversity and scalability of synthetic data.¹⁶ Diffusion models excel at generating high-fidelity tabular and longitudinal data, while LLMs, when fine-tuned on clinical data, produce coherent and contextually accurate clinical narratives. When combined, these models enable multimodal synthetic datasets that reflect the complexity of real-world healthcare, integrating structured data, like diagnoses and prescriptions, with unstructured text, such as progress notes.¹⁷ Moreover, these models can be trained with privacy-preserving techniques such as differential privacy, allowing researchers to simulate large-scale health data while minimising re-identification risks.¹⁸ Synthetic data generation can support research in settings where access to real patient data is limited. It offers a promising path for developing tools and healthcare insights. Like the algorithm-level, data-level approaches are based on an original dataset which could also perpetuate bias and racism within the creation of a synthetic dataset, resulting in harm.

Building on the strengths of these three pathways, a practical and scalable solution could centre on the generation of multimodal synthetic datasets, where both clinical text and structured tabular features are co-generated to reflect the complexity of real-world patient records. This approach provides a foundation for capacity building and methodological development, enabling research without exposing sensitive patient information (e.g., where the original identifiable or sensitive personal primary care data can remain at source). While not essential, federated learning offers a valuable option for collaborative model training across institutions, allowing the use of diverse datasets without centralising raw data. Similarly, TREs can enhance the approach by securely hosting synthetic datasets and co-ordinating federated learning activities, supporting governance, auditability and legal compliance. By prioritising data-level innovation and optionally integrating platform-level and algorithm-level enhancements, this strategy offers a promising pathway to unlock the potential of primary care data. However, given the limitations of each individual approach described above, robust discussion on the ethics and safety of data-level approaches within a New Zealand context is required. 

Māori data governance

Māori data governance is a critical factor in the New Zealand context. The Māori health data governance model, based on the eight pou (pillars) from Te Kāhui Raraunga Māori data governance model, provides a basis for discussion on the technical approaches described above, particularly relating to the development of a synthetic dataset.^11,12 This model incorporates the concept of data sovereignty, which relates to the rights and interests Māori have in relation to Māori data.¹² The first six pou are the most relevant, and some discussion questions follow. The questions within each pou are intended to be a starting point to identify potential risks and benefits of a synthetic data set. Answering these questions and discussions with stakeholder groups, including Māori, will need to occur prior to the development of a national primary healthcare dataset, whatever approach is taken.

• Pou 1: Data capacities and workforce development. How will the synthetic dataset centre iwi/hapū/whānau led solutions? How will the synthetic dataset identify who is experiencing privilege within the health system? How will tikanga and/or kaumātua and Māori health researchers be able to be involved with the development of a synthetic dataset and other technical approaches?
• Pou 2: Data infrastructure. Who will “own” the synthetic data? Where will it be stored? Will the datasets have an “expiry” date?
• Pou 3: Data collection. What will be put in place to ensure that the process of creating a synthetic dataset will benefit Māori (compared to creating harm)? Will a memorandum of understanding be established between iwi/hapū before creating a synthetic dataset?
• Pou 4: Data protection. Will there be any financial gain from the development of new technical approaches to primary care data, and who will benefit from this?
• Pou 5: Data access and sharing. How will the goal of improving Māori health be established prior to creating the synthetic dataset? How will the synthetic dataset be used for direct, tangible and meaningful Māori health benefits? How will research consultation and collaboration occur to identify the potential for harm associated with creating a synthetic dataset?
• Pou 6: Data use and reuse. Linking in with other previously mentioned questions, how will health equity for Māori be planned for and built into the system, and processes be put in place to prevent bias? How will Te Ao Māori, tikanga and mātauranga Māori be part of the creation of the synthetic dataset and other technical approaches? What is the consultation process for creating a synthetic dataset from Māori health data? How will racist algorithms be identified and prevented from being used?

Ethical considerations

The technical approaches described above raise specific ethical concerns but should also direct our attention to bigger philosophical questions; the potential integration of synthetic datasets requires consideration of value prioritisation and paradigmatic differences in data sharing approaches. The main specific ethical concerns relate to re-identification, transparency, bias and fairness. Privacy considerations and requirements may limit the extent to which health data can be shared for research.¹⁹ Although synthetic data has the potential to address privacy concerns that currently hinder data sharing, risks of re-identification remain.²⁰ This means that staying close to the original data will need to be balanced against privacy goals in these datasets. The lack of understanding exactly how AI models generate synthetic data further contributes to questions of representation: can synthetic data represent the realities of healthcare accurately?²¹ Synthetic datasets offer the potential to correct for certain biases, but are also at risk of perpetuating or amplifying biases that exist in the source data.²² Māori datasets are known to have errors, and if used for the development of synthetic datasets will both limit the benefit of synthetic datasets and increase the potential for harm, further reducing equity.^23,24

The distance synthetic datasets create between data and data subjects aims to protect privacy and confidentiality; at the same time, this distance may have unintended negative consequences in terms of fairness and community engagement.²⁵ The distance makes it harder to ensure any benefits derived from the data return to the data subjects. Moreover, when researchers rely on synthetic datasets, there is less incentive to engage with communities, in particular underserved or under-represented populations.²⁵ Overall, to be successful, synthetic datasets will require robust governance, monitoring of data quality, identification and mitigation of potential bias and awareness of potential group harm.^20,21,25 Consideration should also be made that the original real-world data be used to verify findings whenever possible.

Globally, two different stewardship models for health data sharing can be discerned: an open-access model and a governance model. The differences in underlying values and methods are significant enough to consider them as two separate paradigms.

For the open-access model (e.g., TREs), accessibility for researchers is the main priority.² Privacy is protected through de-identification and new avenues, such as the creation of synthetic datasets as described above. Although there is no direct link with patient data, reciprocity and community governance remain important ethical principles. To uphold these principles, UK health research funders (e.g., the National Institute for Health and Care Research and the Medical Research Council) require researchers to include active participation of patients and the public in their research. This has led to research team development that includes patients and the public as co-investigators (e.g., to support study design, data and results interpretation, and dissemination/knowledge transfer). There is a minimum expectation that some research funding is dedicated to help facilitate public participation and a growing expectation from journals such as the British Medical Journal to include statements describing public involvement.

In the governance model, access to data is controlled or restricted. Researchers need to present their plans to a governance group that manages data access. There is less emphasis on de-identification or anonymisation, as the fact that data may be identifiable or that there is a risk of re-identification is considered in the regulatory process. Reciprocity and transparency are prioritised: ensuring benefits from the research go back to the data subjects, understanding and mitigating bias and carefully monitoring outputs from the research are key attention points of the governance work. In the latter model, there is a closer relationship between the data subjects and the researchers. One example of how this governance model can be realised is provided by the recently developed Rakeiora Genomics Platform that is already being applied to whole-genome DNA sequencing data.²⁶

The UK primary healthcare synthetic dataset (OpenSAFELY) has been developed and constructed within the open-access model. However, this does not preclude the use of synthetic data in a governance model. Synthetic data may be useful to help fill data gaps and balance representation in data sets, and communities could be involved in decision-making around the use of synthetic datasets.²⁵ A synthetic national health dataset based on routinely collected primary healthcare data, leveraging these sophisticated methods of leaving data in situ and providing improved access, could be an important resource for “public good” research and public policy creation.^27,28 Encouraging more systematic coding of disease within primary healthcare records (for example using SNOMED-CT codes), would permit more straightforward and robust analyses, particularly for chronic conditions.

Primary healthcare data access in Aotearoa New Zealand

The New Zealand public recognise there are benefits to sharing their healthcare data and are willing to share anonymised health information for research purposes, with appropriate safeguards in place.^29,30 A discussion on how to best share primary health data for research in New Zealand is needed. Māori data governance frameworks are ideally positioned to form the basis of such a discussion and are easily adaptable to consider new technological methods.¹¹ In a governance model, a national data infrastructure could be created with strict governance over the data and access. In an open-access model, datasets could be de-identified or synthesised, with the aim of sharing primary healthcare data as widely as possible for research. These are different models that require different infrastructures.

Regional networks with capacity to link primary healthcare data for research are already operational in some parts of New Zealand, such as the SPCRN.⁴ PHOs are well placed to access primary healthcare data as they usually have some capacity to do this already. A first step is to empower all PHOs to permit health data access for research. The next step is to collate regional primary healthcare data into larger groupings, such as the agreement made by the Te Waipounamu data collective.⁸ A national primary healthcare data–sharing agreement between New Zealand PHOs and a central organisation responsible for the data governance and administration has potential to improve the research capacity and capability of primary health researchers. From here, considerations may be given to consider whether following the UK example of developing TREs and/or a synthetic dataset is appropriate for New Zealand.

Conclusion

New Zealand lacks an integrated national primary care research approach that can interrogate routinely collected primary healthcare data and is missing out on the potential benefits of research using these data. Some regions have taken steps to improve data access for researchers; however, access across New Zealand is patchy, and no national collection exists. International experience suggests various technical methods can be used to improve researcher access to primary healthcare data, be it platform-level, algorithm-level or data-level, or using a combination of these.

We have described two stewardship models for primary healthcare data. The governance model requires strict control, governance and oversight. The alternative is a more open-access, privacy-preserving model which could use methods such as the creation of synthetic datasets to provide easier access for researchers to develop and test models within a trusted research environment.

The advantages and disadvantages of these technical approaches and stewardship models need to be carefully considered in the New Zealand context to ensure any action taken to widen primary healthcare data access for research includes Māori data governance and upholds ethical research standards. Broadening the conversation to include patients and the public is needed to understand the impact of developing a national primary healthcare research network and the use of new technical approaches to increase researcher access to primary healthcare data.