Maintaining patient trust as artificial intelligence’s role in healthcare grows

Patient trust in health institutions and health professionals has long been recognised as key to effective healthcare delivery.^1,2 While definitions and measures of this complex concept have varied, it is theorised that a patient’s level of trust in health professionals is largely based on the domains of competence, compassion, privacy, confidentiality, reliability and communication.² It is also considered that trust in healthcare is tied to expectations that health professionals will prioritise the patients’ best interests while adhering to the principles of beneficence, fairness and integrity.³ Patient trust in their healthcare professionals is thought to be distinct from patient satisfaction, and is a stronger indicator of the relationship between the patient and their clinician, defining a patient’s expectations for their clinician’s motivations.⁴ A patient’s level of trust in healthcare institutions, influenced by the media and the broader atmosphere of social trust, is also important to consider in the framing of individual patient–clinician rapport.² Studies have shown that patient confidence in health professionals to act in their best interest is associated with more positive health behaviours, improved quality of life and reduced symptoms.^5–8 Conversely, low levels of trust or mistrust in health professionals and services are linked to non-engagement in services, low compliance with care recommendations and poorer health outcomes.^9,10

Trust in health institutions is easily lost, and once damaged can be hard to repair. Evidence for predictors of patient trust suggests that demographic characteristics such as gender,¹¹ ethnicity^11–14 and age¹¹ may play a role in one’s level of trust; however, other individual factors such as past personal experiences with healthcare appear to be a strong and consistent predictor of trust levels.^11–13 Research in Aotearoa New Zealand has demonstrated that any past experiences of racism, regardless of the source, negatively impact patients’ confidence and trust in their general practitioner (GP).¹⁵ COVID-19 provided the world with a sobering example of the fundamental nature of people’s trust in health professionals, systems and agencies in determining care-seeking behaviours and vaccine uptake.^1,16 Critically, trust in key institutions among New Zealanders continues to decline in the post-COVID pandemic era,¹⁷ with just over half of New Zealanders (53%) reporting in 2023 that they trust the Aotearoa New Zealand health system to give them the best treatment.¹⁸

It is worth noting that declining trust in health professionals and healthcare institutions is not occurring in isolation. Declining trust in public institutions has been documented in democratic countries internationally in recent years and may be attributed to numerous uncertainties including the COVID-19 pandemic, rising inflation and economic downturn, wars, major geopolitical upheavals and major weather events.¹⁹ These issues, while significant in their own right, may be exacerbated by the visibility of political polarisation, misinformation and disinformation and political disengagement.¹⁹ A complexity of individual and societal factors may therefore contribute to people’s sense of government inability to respond to complex policy issues—including healthcare-related issues—appropriately and in people’s best interest.

Legitimate use of personal data is considered to be one major driver of trust in public institutions.¹⁹ While already used routinely for research and quality improvement, and to inform planning and policy, secondary use of patient data and its impact on patient trust is under renewed spotlight with the rapid advancement and public awareness of artificial intelligence (AI) in the healthcare setting. For the purpose of this viewpoint, the Organisation for Economic Co-operation and Development (OECD) definition of AI is used:

“An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.”²⁰

AI is thought to have the potential to transform medical practice with applications spanning nearly all aspects of healthcare, from risk prediction to treatment monitoring.²¹ It also has the potential to improve equitable outcomes by increasing access and affordability of medical care and addressing biases.²² However, to fully realise this potential, it requires the availability of large amounts of contextually relevant patient data to train and test new AI tools,²³ for which individual patient consent has often not been obtained. While many AI tools are developed through research with well-established approval and ethics procedures, health services are increasingly responsible for deciding whether to provide access to patient data for AI training and testing in the real-world setting.²⁴ They also must decide whether and how the resultant AI tools should be integrated into clinical care.²⁵

In this rapidly evolving environment, building and maintaining patient trust is of growing importance as it is not only essential for permitting AI development and implementation to continue, but it also has implications for AI’s potential to positively impact healthcare provision and patient outcomes once implemented. Moreover, public levels of knowledge about AI appear to be low,^26,27 and thus trust in health professionals and healthcare institutions to be good stewards of their sensitive health information and care in the face of AI-based health technologies is critical.

In terms of secondary use of patient data for AI, emerging evidence suggests that while healthcare consumers are supportive of the use of their data in certain circumstances, this acceptance comes with conditions: that the data are used primarily for public benefit rather than commercial gain; that they are well informed and given a choice about the use of their data where possible; and that their data are kept secure and their privacy maintained through de-identification when shared with external parties.^26–28 In short, patients want to be able to trust that their data will be used ethically and responsibly, and will be kept safe.

Appropriate regulation and responsible use of AI are also considered key drivers of trust in public institutions.¹⁹ Regarding the integration of AI into clinical practice, research shows that while patients anticipate benefits such as improved diagnostic accuracy, increased efficiency, reduced errors and reduced workload for health professionals, there are also a number of concerns and doubts.^27,29 These include a reduced emphasis on clinicians’ own expertise and agency, potential for biases and increased health inequities, care interactions lacking human compassion, transparency and communication, the inappropriateness of AI for some situations and a lack of confidence in AI’s readiness and reliability for effective and safe healthcare delivery.^27,29

The potential for AI to improve healthcare can only be achieved with patient and public trust that their health services and professionals will use their data and apply AI technologies safely, ethically, effectively and appropriately. The remainder of this paper will discuss our own explorations into patient perspectives on the secondary use of personal health information and the implementation of AI in healthcare in Aotearoa New Zealand, with a focus on the role of trust in healthcare consumers’ comfort with the development and implementation of these technologies. This is followed by a set of recommendations to researchers, policymakers and health service providers on how to prioritise and maintain patient trust in this rapidly evolving environment.

AI in healthcare and patient trust: evidence from Aotearoa New Zealand

The evidence base for patient or consumer perspectives on the use of patient data for AI development and integration into healthcare in Aotearoa New Zealand is small but growing.^30–32 To date, our group has conducted research to explore consumer views on this topic and on the secondary use of data more broadly, with some findings already published.^32,33 Findings have been used to inform national guidelines and policies in our national public health service (Health New Zealand – Te Whatu Ora) including in the development of the AI Governance Framework used by the National AI and Algorithm Expert Advisory Group (NAIAEAG).³⁴

Two of the studies involved individual interviews with potential AI end users (adult health service users and health professionals) of Health New Zealand – Te Whatu Ora (combined n=63) between October 2021 and February 2024. The interviews were semi-structured and scenario-based, as prior work with consumers revealed differing levels of literacy with regards to AI. Interviews were recorded, de-identified and transcribed. Both studies received ethical approval (New Zealand Health and Disability Ethics Committee reference numbers 20/NTA/2; 2023 EXP 18411). Recruitment methods varied based on COVID-related restrictions but included clinicians approaching patients, sharing the study flyer through clinical and academic networks and posting about the study on relevant patient forums. Analysis methods were specific to each study and included thematic analysis³⁵ and the rapid assessment process.³⁶ During the interviews the participant was read a series of hypothetical scenarios in which de-identified patient data were collected to inform the development of an AI tool and the tool subsequently used in clinical practice. Participants were asked to provide their thoughts on the scenarios and potential issues.

Participants across both studies ranged in age from 22 to 77 years and included representation of priority ethnic groups (including 20.6% Māori, 6.3% Pacific and 7.9% Asian). Our interviews revealed several key themes in which maintaining patient trust was central to participants’ overall sentiments about AI in healthcare. In particular, participants felt that the following conditions must be present in order to maintain patient trust in an AI-enabled environment:

• The sharing of patient data for AI development is only allowed where the primary benefit is to the Aotearoa New Zealand public.
• Health institutions have strong oversight and governance processes over the continuum of AI development and implementation, with health professional involvement throughout.
• Strong data protection and data governance procedures are in place and prioritised.
• Patients are given choices around AI’s role in their care and, where possible, asked for consent to the use of their data for AI development purposes, regardless of how benign the data might appear.
• Health professionals should be the “face” of this, meaning they must be willing to initiate conversations with individual patients about the use of their data and the use of AI in their care.
• Full transparency must be provided around any use of patient data in AI development including who will be involved, how data will be used and how data are being protected.
• External partnerships should be carefully considered. Universities and Aotearoa New Zealand–based organisations are seen as more trustworthy AI development partners than commercial companies or overseas institutions.
• There is Māori representation in work to develop or train AI tools, and governance over AI implementation in healthcare.
• AI is not expected to replace the human connection between health professionals and patients, nor should it over-ride human expertise and contextual knowledge.

Throughout these interviews and in line with the broader evidence on patient trust, it was apparent that participants place high value on their relationships with health professionals as a proxy for trust in the health institutions. Additionally, consistent with recent studies done elsewhere on patient perceptions of AI,^27,29 the rapport and quality of interactions with health professionals was seen as vital and superior to potential interactions with AI-based technologies. For this reason, AI was not viewed as a trustworthy replacement for clinicians possessing the “human touch” and should instead be viewed as a tool in the clinician’s toolbox to improve accuracy and efficiency, and allow them to spend more time providing patient care.

While the work to date has provided valuable information, more evidence is needed to understand how patient levels of trust vary under different circumstances. As we interviewed patients belonging to a variety of groups, it became clear that group-level emphases were nuanced (for example, patients with rare conditions had different views on international involvement than patients with mental health conditions). More research to clarify and deepen our understanding of consumer views on these issues is needed. It was apparent in this research that understanding and perceptions of AI were changing over time; therefore, as AI becomes more commonplace within society, it is likely that patient trust in these technologies will evolve. Ongoing engagement with end users will ensure health institutions can adapt to accommodate the advances in AI while maintaining patient trust.

Recommendations

These findings have led to the following recommendations for maintaining patient trust in health institutions and their health professionals as AI’s role in healthcare grows. Importantly, there must be collaboration between government, researchers and clinical leadership to ensure that health professionals are equipped to support patients and prevent AI-related concerns from interfering with quality healthcare provision.

Recommendation 1: Ensure there is a culture of transparency around AI in health, including secondary data use and AI involvement in healthcare. Transparency means that sufficient information is available to all interest holders, at all stages of the AI lifecycle (from development and design to evaluation and monitoring), to facilitate meaningful public discussion and debate on how an AI tool is designed and deployed. This includes accurate information about the potential limitations and risks of the technology, the nature and extent of patient data use and algorithms used for training and testing.²⁵

a.      Due to the complexity of AI integration and the breadth of datasets needed for AI development/training/testing, the principle of transparency must be adopted and led nationally rather than be the responsibility of individual health professionals and services.

b.      Transparency must be coupled with education to ensure adequate general AI literacy is achieved across the population through combined governmental (as stewards of the healthcare system) and sectoral approaches, so that the public has a clear understanding of not just what AI is but also how it may be used in their healthcare.

c.       Health professionals must receive the necessary training to ensure they have the language and skills to communicate and be transparent with their patients around AI.

Recommendation 2: Support and enable good governance over AI in health in Aotearoa New Zealand following best-practice guidelines and the latest evidence. This will ensure that ethical, legal and privacy considerations are met (see^37–39), tools are reliable and safe and development and use of AI in health is appropriate.

a.      Governance must include representation of end users, including health professionals and patients.

b.      Good governance must be enabled to establish or update guidelines, policies and regulations (e.g., ethics guidelines) according to emerging international best practices (e.g., from the World Health Organization and other global agencies^20,40).

Recommendation 3: Ensure that AI is developed and used in health for the benefit of the Aotearoa New Zealand public. This includes that its development and use are safe and culturally and clinically appropriate, as well as for the primary purpose of improving health outcomes rather than commercial gain. This is particularly important for AI tools developed outside of Aotearoa New Zealand.

a.      Use of patient data for AI development and testing should be only endorsed where there is clear benefit to New Zealanders. This includes that, where the sharing of data occurs with external or overseas organisations, there is sharing of benefits in return.

b.      Health data from New Zealanders should be protected as taonga and commitment to Māori data sovereignty must be maintained.⁴¹

c.       AI use in care should benefit those who need it most and not disadvantage population groups, nor be a blanket replacement for in-person care.

Conclusions

AI’s potential contributions to healthcare cannot be realised without a high degree of patient and societal trust that their health services and professionals are acting in their best interests. The rapid emergence of AI in health has not been coupled with consistent increases in AI literacy among the general public, so it is vital that patients can trust their institutions to make decisions on their behalf through good governance over patient data for AI development and implementation of AI in care settings. Alongside governance, transparency in the use of data and AI will be fundamental to maintaining and building trust in health institutions. Our research has shown that with good governance and transparency we can help to ensure that health services are equipped to support health professionals and patients through the safe and timely implementation of AI into care.