Accuracy of rheumatoid arthritis diagnosis coding in primary care: a validation study

Rheumatoid arthritis (RA) is a chronic autoimmune disease characterised by systemic synovitis commonly affecting the hands, wrists and feet, with a multifactorial aetiology: genetic susceptibility, e.g., HLA-DRB1 allele variants, and environmental factors including smoking, obesity and exposure to air pollutants contribute to disease risk.^1,2 Globally, 70% of people living with RA are women, and 55% are older than 55 years. Untreated RA can severely damage joints and surrounding tissue, impact the heart, lung and nervous system and cause chronic pain and make it difficult to perform daily activities. This significantly impacts quality of life and an individual’s morbidity, and has the potential to place further strain on the healthcare system of Aotearoa New Zealand.¹ RA also carries increased mortality risk: a 2025 study from Canterbury reported all-cause standardised mortality ratios for prevalent RA of 2.01 (95% confidence interval [CI] 1.85–2.19) between 1 January 2006 and 31 December 2008, and 1.87 (95% CI 1.69–2.06) between 1 January 2011 and 31 December 2013.³

Accurate identification of individuals with RA within routinely collected healthcare datasets underpins disease surveillance, service planning and clinical research, thereby helping reduce morbidity and mortality by allowing accurate measurement of disease burden, earlier recognition of preventable complications, evaluation of treatment effectiveness and targeted intervention for high-risk subjects.

However, the validity of RA case definitions within real-world datasets is variable. Diagnostic coding can be inconsistent, disease-modifying antirheumatic drug (DMARD) prescribing is not specific to RA and the extent of rheumatology specialist involvement is often unclear. Misclassification remains a significant challenge. For example, a 2011 American study reported positive predictive values (PPVs) for diagnosis-code-based RA algorithms ranging from 55.7% (95% CI 46.8–64.4) to 66.7% (95% CI 55.5–76.6), improving to between 86.2% (95% CI 74.6–93.9) and 88.9% (95% CI 76.0–96.3) when DMARD prescription criteria were included—both being compared to the gold standard of a diagnosis of RA by a rheumatologist.⁴ These findings highlight the need for validated, context-specific approaches to RA case identification.

In Aotearoa New Zealand, primary health organisations (PHOs) such as the Pinnacle Midlands Health Network hold large volumes of routinely collected subject data, yet no validated algorithm exists to identify RA cases within these datasets. This study aims to validate the accuracy of SNOMED-coded RA diagnoses in primary care by linking these primary care records to rheumatology specialist encounters, DMARD treatment history and RA-specific serology. Through a structured, stepwise validation pathway, we proposed a scalable and transparent method to improve the reliability of RA case definitions in real-world datasets and to support future epidemiological and health services research in Aotearoa New Zealand.

Methods

Study design, setting and objective

This was a retrospective diagnostic validation study assessing the accuracy of RA case identification in primary care within the Midlands Region of Aotearoa New Zealand—Waikato, Lakes, Bay of Plenty, Taranaki and Tairāwhiti.

Study data sources and population

Study subjects were those with a SNOMED diagnostic code for RA recorded between 1 January 2018 and 31 May 2025, and were identified from the Pinnacle Midlands Health Network database. Subjects were identified using National Health Index (NHI) numbers. Extracted variables for each subject included: age, sex, ethnicity, deprivation quintile, practice region, date of first coded RA diagnosis, rheumatology specialist involvement (linkage to the M70 national rheumatology service code) and prescribing history, including DMARDs.

RA-specific serology—rheumatoid factor (RF) and anti-cyclic citrullinated peptide (aCCP) autoantibodies—data for individuals with NHI numbers were obtained from Pathlab, the regional medical laboratory service covering Waikato, Lakes and Bay of Plenty. Individuals from Tairāwhiti and Taranaki were captured in these datasets if travelling into Pathlab regions for serological testing. Study subjects were cross-linked using their NHI numbers. The Waikato Hospital clinical workstation database, and primary care records accessible through it, were accessed where required to supplement incomplete data and review clinical records

Inclusion and exclusion criteria

All subjects with a recorded SNOMED diagnostic code for RA in the Pinnacle Midlands Health Network database during the study period were eligible for inclusion. Subjects from Taranaki were excluded from validation steps requiring access to Waikato Hospital records due to data access limitations.

Case definition and validation criteria

RA case validation was determined through a structured, hierarchical analysis.

1)      Subjects with documented rheumatology specialist involvement (M70 service code linkage)

Subjects in this sub-group with the prescription of a DMARD with a start date recorded from 1 January 2017 onwards were considered to have RA.

Subjects in this sub-group who did not meet this criterion underwent independent review of their full medical records by a final-year medical student, merging data from the Pinnacle PHO, Pathlab, Waikato Hospital clinical workstation and primary care databases. Subjects miscoded as not being prescribed a DMARD despite having a recorded DMARD start date on or after 1 January 2017, as well as subjects with a DMARD start date prior to 1 January 2017, were classified as having RA unless their full medical record indicated an alternative rheumatological condition they were receiving DMARD treatment for, e.g., psoriatic arthritis, or showed that a DMARD had been trialled before RA was excluded. Subjects with serological evidence of RA, and/or subjects with clinical evidence of RA described by two experienced rheumatologists—10 or more years of consultant clinical experience—were also considered to have RA. Equivocal cases were reviewed by these rheumatologists, and a consensus was reached to classify these subjects as either having RA or not.

Serological evidence of RA was determined using reference ranges established by these rheumatologists for RA-specific autoantibodies:

• Positive RA-specific serology—RF >50IU/mL, and/or aCCP >100IU/mL
• Possible seropositivity— RF 25–50IU/mL, and/or aCCP 20–100IU/mL
• Negative serology—RF <25IU/mL, and/or aCCP <20IU/mL

2)      Subjects with no documented rheumatology specialist involvement (no M70 service code linkage)

Subjects in this sub-group with the prescription of a DMARD with a start date recorded from 1 January 2017 onwards were considered to have RA.

For subjects in this sub-group without DMARD exposure, RA validation relied on serological evidence—reference ranges have been outlined above. Subjects with positive serology were considered to have RA. Subjects with possible seropositivity were reviewed by two senior rheumatologists, and a consensus was reached on the likelihood of the subjects having RA. Subjects with negative serology were considered to not have RA.

3)      Subjects with no documented rheumatology specialist involvement (no M70 service code linkage), no DMARD exposure and no serological data

Given the limited data available for these subjects, a random sampling validation strategy was implemented to closely analyse a smaller subset (excluding Taranaki subjects). Excel’s “RAND()” function was applied to the sub-group of subjects, and the first 50 cases following randomisation were chosen for analysis. Again, an independent review of their full medical records was undertaken to look for evidence of RA. Equivocal cases were reviewed by two senior rheumatologists as above, and a consensus was reached on the likelihood of these subjects having RA. The proportion of confirmed RA cases within this randomised subset of 50 subjects was used to estimate the expected prevalence of true RA in the broader sub-group.

Additional internal validation assessment

To assess the accuracy of DMARD-based RA case classification, an internal random-sampling validation process was undertaken. Fifty subjects with documented rheumatology specialist involvement and current DMARD treatment and 50 subjects without documented rheumatology specialist involvement but with current DMARD treatment were randomly selected (excluding Taranaki subjects). Randomisation was performed using Excel’s RAND() function, and the first 50 cases following randomisation for both groups were chosen for analysis. Again, independent review of full medical records was to classify subjects as having RA or not, and equivocal cases were reviewed by the two senior rheumatologists with a consensus formed to classify these subjects as either having RA or not.

Statistical methods used

All data management and processing were performed in Microsoft Excel and IBM SPSS Statistics version 31 (IBM Corporation, New York, United States of America [USA]). We calculated PPV as the percentage of the 3,831 subjects who met the gold-standard criteria for validation as described above.

Ethical considerations

Ethics approval for this study was obtained from the Northern B Health and Disability Ethics Committee (reference: 2024 EXP 20314).

Results

View Figure 1.

A total of 3,831 subjects had a SNOMED diagnostic code for RA recorded in Pinnacle PHO primary care data between 1 January 1 2018 and 31 May 31 2025.

There were 1,567 subjects (40.90%) who had documented rheumatology specialist involvement with linkage to an M70 national rheumatology service code. Of these, 1,473 subjects (94%) had evidence of current or previous DMARD treatment. After review, 1,468 were confirmed to have RA, while five were receiving DMARD treatment for other rheumatological conditions, e.g., psoriatic arthritis, or had been trialled on a DMARD and later found not to have RA. The remaining 94 of these subjects (6%) had no recorded DMARD treatment. Following full medical record review, 12 were confirmed to have RA based on serological and/or clinical evidence described by a senior rheumatologist, and 82 were classified as not having RA.

A total of 2,265 subjects (59.10%) had no documented rheumatology specialist involvement with no linkage to an M70 national rheumatology service code. Of these, 1,747 subjects (77.2%) were receiving DMARD treatment with a start date recorded from 1 January 2017 onwards. These subjects were classified as having RA and were not scrutinised further. The remaining 517 subjects (22.8%) were not on DMARD treatment with a start date recorded from 1 January 2017 onwards. Seventy-five had available serological results: 16 were seropositive and classified as having RA, 50 were seronegative and classified as not having RA and nine had possible seropositivity that required further review by two senior rheumatologists. A consensus was reached on these nine subjects, with one classified as having RA and six as not having RA. Two remained inconclusive. The final 442 of these 517 subjects had an unknown serological status. A randomised sample of 50 of these subjects had their full medical records scrutinised, with equivocal cases undergoing further review by two senior rheumatologists: seven of the 50 (14%) were classified as having RA, and 43 (86%) as not having RA. Applying these proportions to the wider sub-group, an estimated 62 of the 442 subjects (14%) were expected to have RA, and 380 (86%) were expected not to have RA.

The additional internal validation assessment of 100 DMARD-exposed subjects confirmed RA in 44 of 50 subjects (88%) with documented rheumatologist specialist involvement (linkage to M70 national service code) and current DMARD treatment, and 40 of 50 subjects (80%) without rheumatology specialist involvement but with current DMARD treatment. The remaining cases were reclassified as alternative diagnoses or insufficient evidence for RA following full medical record review.

Discussion

In this retrospective diagnostic validation study, 3,306 of the 3,831 subjects with a SNOMED-coded RA diagnosis were confirmed to have RA using a structured validation pathway incorporating rheumatology specialist involvement, DMARD exposure and RA-specific serology, with 523 (13.7%) classified as not having RA and two (0.05%) remaining inconclusive. This corresponds to a PPV of 86.3% with 95% CIs of 85.2–87.4%, indicating that most RA-coded diagnoses within the primary care dataset reflect true disease. This is consistent with international validation studies—for example, a USA study in which diagnosis code–based RA algorithms achieved PPVs of 55.7–66.7%, which improved to 86.2–88.9% when DMARD prescription criteria were included, utilising a gold standard of a diagnosis of RA by a rheumatologist to compare against.⁴

Confirmation rates for RA were highest among subjects with documented rheumatology specialist involvement: 1,480 of these 1,567 subjects (PPV 94.4%) were validated as having RA. These findings align with studies from Western Australia and Sweden, which reported PPVs of 92% and 91% respectively, when the PPVs of RA-diagnosis codes were validated against a gold-standard rheumatologist-reported diagnosis.^5,6 This reflects the central role of specialist-led accurate diagnosis and treatment in RA care.

Despite high overall PPV, 13.7% of RA-coded subjects did not meet validation criteria. Misclassification was concentrated among subjects without rheumatology specialist involvement, without DMARD exposure and without serological data. This sub-group accounted for most false-positive diagnoses: 380 of the 442 subjects (86%) were classified as not having RA. Analysis of a randomised sample of 50 subjects from this sub-group revealed that many had alternative diagnoses such as osteoarthritis or palindromic rheumatism that were miscoded as RA, or provisional RA diagnoses that persisted in primary care records. These findings illustrate the limitations of relying solely on diagnostic codes and the importance of integrating multiple data sources to improve accuracy.

Within Canterbury, age-standardised RA prevalence rates of 442.65 and 448.66 per 100,000 population were reported for the periods 2006–2008 and 2011–2013, respectively.³ These are more than double the age-standardised global prevalence rate of RA of 208.8 cases per 100,000 population reported in 2020.⁷ Secondary data suggest that the Midlands may share features associated with the elevated RA prevalence observed in Canterbury, though prevalence was not directly estimated in this study. Regional differences in population-level risk factors previously associated with RA—smoking, obesity, exposure to air pollutants and animal dander allergies—may contribute to both regions’ RA prevalence rates.^1,8 The presence of large agricultural sections in both regions may also play a role; however, this remains speculative and warrants further investigation. Collectively, these findings raise the possibility that age-standardised RA prevalence in Aotearoa New Zealand may be elevated at a national level compared to current global estimates, which again requires further investigation.

This study has several strengths, including a large primary care cohort encompassing the Midlands Region, which enhances external applicability of findings across diverse urban and rural populations. Use of the NHIs enabled integration of multiple data sources—primary care records, rheumatology specialist encounters and regional Pathlab serology—reducing reliance on a single data stream and enhancing RA validation, with equivocal cases independently reviewed by two senior rheumatologists. However, limitations also exist. The retrospective use of routinely collected data may be incomplete or inconsistently recorded and may introduce selection bias. A subset of 442 subjects lacked serological data, DMARD exposure or specialist involvement, limiting diagnostic certainty; randomised sampling was used to estimate misclassification, although residual error is likely. Exclusion of Taranaki subjects from this randomised sampling may limit regional generalisability, and our study focussed on PPV without assessment of sensitivity, meaning true RA cases not coded in primary care records were not captured. Other subjects lacked serological data, and although DMARD exposure and rheumatology specialist involvement were used as pragmatic proxies for RA diagnosis as supported by prior literature, not all cases underwent full manual review and diagnostic variability among rheumatologists is inevitable in real-world practice, leading to potential misclassification.^4–6 Our additional internal validation demonstrated high confirmation among DMARD-exposed subjects with specialist involvement (88%), but lower confirmation without specialist involvement (80%), indicating modest residual misclassification. Additionally, false-negative RA cases were not systematically assessed, which may have influenced final estimates. This study focussed on SNOMED coding of RA within the dataset, and the findings, while positive, cannot be generalised to other conditions recorded in the dataset.

These findings have important implications for clinical practice, epidemiology and health system planning in Aotearoa New Zealand. Misclassification of RA in primary care records may distort estimates of disease burden, treatment coverage and ethnic or geographic inequities in access to rheumatology care. Such misclassification likely reflects not only limitations of SNOMED diagnostic coding, but also broader system-level factors: constrained specialist access, variable clinician familiarity with early or atypical RA presentations and reliance on provisional diagnoses in busy primary care settings. Updating and refining the SNOMED coding system, alongside further research into and addressing of these systemic factors, is essential to improve accuracy of RA identification and support reliable health data for clinical and research purposes.

In conclusion, SNOMED-coded RA diagnoses in primary care records demonstrate high PPV when evaluated using a structured, hierarchical validation pathway. Adoption of validated case definitions is crucial to support accurate epidemiological research, health service planning and equitable RA care in Aotearoa New Zealand.