Validation of the Edinburgh Postnatal Depression Scale (EPDS) as a screening tool for postnatal depression in Samoan and Tongan women living in New Zealand

Postnatal depression (PND) is a serious public health issue and more so in women of Pacific ethnicity in New Zealand (NZ).1,2 Pacific women have a higher fertility rate, most are untreated in the community; and depression is higher in NZ-born women than those born in the Islands.3-5PND causes major maternal morbidity resulting in dysfunctional relationships and with effects that continue to affect the woman's children—deficits in social, psychological, and cognitive domains and an increased risk of suffering from child abuse.6,7The prevalence of PND from an aggregation of worldwide studies averages 13% (of all pregnancies)8 but the only study specifically on Pacific women found a prevalence of 16.4%.1The study cohort of 1376 Pacific women, in the Pacific Island Family Study (PIF study), found a statistically significantly difference in prevalence rates of 30.9% in Tongan women and 7.6% in Samoan women.The PIF study used the Edinburgh Postnatal Depression Scale (EPDS), a 10-item self-report questionnaire that was developed in 1987,9 and has been translated into many languages and used in many prevalence studies. Even though the EPDS had also been used in two other NZ studies,10,11 it had not been validated in Pacific or NZ populations. The PIF study administered the EPDS as an interview and "a score above 12 is widely used to indicate the presence of probable depressive disorder".1The EPDS has been validated in more than 25 different ethnic groups and populations.12-14 A review of 37 validation studies of the EPDS had showed a highly variable sensitivity from 34%-100% and a specificity of 44%-100%,12 therefore supporting several recommendations that validation precede clinical use in culturally diverse populations.9,15The purpose of this study was to validate the EPDS in Samoan and Tongan women as a pre-requisite to repeating a prevalence study. These two population groups make up 65% of all Pacific women residing in New Zealand.16Materials and Methods Background—This prospective cohort study was approved by the Northern "X" Ethics Committee and was conducted at Middlemore Hospital in Auckland, New Zealand. It is the referral hospital of the Counties Manukau District Health Board (CMDHB) with a catchment population of 500,000 that includes about 36% of all 250,000 people of Pacific ethnicity in NZ. Questionnaire—The EPDS is a commonly used screening tool and rates each of the 10 items on a four-point scale (0-3), giving a maximum score of 30. A score of ≥13 has been used in previous prevalence studies signifying serious PND whereas women with scores of 10 to 12 were considered to have mild depression. A woman with a score of 0 to 9 was considered not depressed. The EPDS has been found to be acceptable to women17 and a useful tool in cross-cultural research on depression.18 The EPDS was translated into the Samoan and Tongan languages and then independently back-translated, by a professional translation service. The translated versions were checked by clinical researchers AE (fluent in Samoan) and SF (fluent in Tongan) for appropriateness of language and meaning. Agreement between depression resulting from the English and translated versions were also tested using a Kappa Statistic. Each translated and English version was then piloted by five Samoan and five Tongan women who were fluent in their language. Sample and data collection—The sample size was determined from published validation methodology for the EPDS.19,20 Names and contact details of Samoan and Tongan women scheduled to deliver the following month were communicated to the research team comprising both a Samoan and Tongan researcher. Excluded from the study were women who were critically ill, had a stillbirth, serious complications in pregnancy or delivery and who were unable to provide informed consent. Women were initially contacted by posted information followed by a phone call. Interested women were recruited in a clinic or at their home. The women could choose to complete the EPDS in English or in their own language and were not offered any assistance in completing the questionnaire. The questionnaires were completed between 4 and 7 weeks after delivery. An interview was then arranged with one of two psychiatrists who were blind to the EPDS scores and who had received accredited training in the use of the World Health Organization Composite International Diagnostic Interview (WHO-CIDI v3). The interview was completed within 4 weeks of completing the EPDS. Psychiatrist SF who was fluent in the Tongan language interviewed Tongan women and SW who was semi-fluent in Samoan interviewed the Samoan women. Interpreters were provided where requested. Women who were diagnosed with serious depression were referred to the Maternal Mental Health Service. Statistical analysis—The raw data was entered into an Excel Spreadsheet (Microsoft Corporation), then analysed using Stata v8.0 software (Stata Corporation, Texas, USA). SAS v9.1 software was used to analyse the CIDI data and diagnoses. Cronbach's coefficient alpha was calculated to estimate the reliability of the EPDS by determining its internal consistency or the average correlation of items within the EPDS. A value of 0.70 is considered an acceptable level for consistency.21 Kappa statistics was used to measure the level of agreement between the EPDS and CIDI standard. A Kappa of >0.6 is an indication of substantial agreement and >0.8 is an indication of an almost perfect agreement.22 The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) as well as the receiver operating characteristics (ROC) were used to determine the global performance of the EPDS against the CIDI and the best cut-off points in predicting PND. Results During the study period, February 2009 to June 2010, a total of 170 women (85 Tongan and 85 Samoan) completed the study and their characteristics are compared in Table 1. There was no significant difference between the Samoan and Tongan women with regards to age, parity, country of birth and mode of delivery. However, 51% of Tongan women completed the Tongan (rather than the English) questionnaire compared to 28% of Samoan women completing the Samoan questionnaire. Table 1. Characteristics of the Samoan (n=85) and Tongan (n=85) women participants Variables Tongan No. (%) Samoan No. (%) P value (df, χ2) Age groups ≤19 20-24 25-29 30-34 35-39 ≥40 Mean (SD) 6 (7) 16 (19) 25 (29) 19 (22) 16 (19) 3 (4) 28.9 (6.38) 1 (1) 20 (24) 19 (22) 23 (27) 17 (20) 5 (6) 29.9 (6.6) 0.22 (5-7.00) Parity 1 2 3 4 ≥5 Not known 23 (27) 18 (21) 15 (18) 10 (12) 18 (18) 1 (1) 24 (28) 17 (20) 19 (22) 10 (12) 14 (16) 1 (1) 0.97 (4-0.54) Birthplace NZ Pacific Islands 26 (31) 59 (69) 24 (28) 61 (72) 0.64 (1-0.22) Delivery Normal Caesarean Instrumental 70 (82) 14 (17) 1 (1) 64 (75) 15 (18) 6 (7) 0.89 (2-4.8) df=degrees of freedom; χ2=Chi squared. Are the items of the EPDS internally consistent?—The Cronbach's alpha for all of the EPDS language versions and English version reached acceptable levels of reliability with adjusted overall alpha values of 0.86. The English version was consistent and there was little difference observed between Tongan and Samoan women. The EPDS scores—About 1 in 5 (19%) of both Tongan and Samoan women had an EPDS score of 10-12. Using the cut-off points as recommended in the original EPDS developed by Cox JL et al9 the prevalence of serious depression (EPDS score ≥13) in this study would be 16.6% and for all depression (EPDS score ≥10) would be at 35.9%. Serious depression was observed in 19% of Samoan and 13% of Tongan women with an EPDS score of 13 or more. The average EPDS score was 8.4 (SD of 5.2), Samoan average was 8.7 (SD 5.1) and Tongan was 8.1 (SD 5.1). The CIDI findings—Of the 170 women interviewed, 36 (21.2%) had a positive CIDI, of which there was an equal number of Samoan and Tongan women. Of those women, 29 (17.1%) women had serious depression by interview (15 Samoan and 14 Tongan). How well does the EPDS compare with the CIDI diagnoses?—The range of EPDS scores for women identified with a serious depressive disorder ranged from 5 to 24 with a median of 15 for both groups. A clear pattern of higher EPDS scores for those with diagnosed disorders was shown. The EPDS median for those with no depression and mild depression was similar in the Tongan women whereas for the Samoan group, the EPDS median was similar for serious depression. 62% of women with serious depression diagnosed by CIDI were identified by the EPDS. If only serious depression was considered, the EPDS compared to the CIDI yielded a Kappa of 0.57, which indicates mild agreement. The prevalence of any diagnosed depression resulted in mild agreement (Kappa >0.5) for Tongan, Samoan and the combined results. A comparison between serious depression identified by the EPDS with the CIDI for serious depression resulted in mild agreement for both Tongan (Kappa = 0.58) and Samoan women (Kappa = 0.56). The area under the ROC was equal to 0.8948 and 0.8308 for Samoan and Tongan women respectively (Figure 1).The probability that a randomly selected Samoan or Tongan woman with depression has a higher EPDS score than one selected without depression is more than 80%. The EPDS score that gave the highest sensitivity (79%) and specificity (76%) for the Tongan women was ≥10. The best score for the Samoan women was ≥11, where both the sensitivity and specificity was 80% (Table 2). An EPDS score of ≥10 gave a PPV of 46% and NPV of 93% for the combined EPDS in all three languages. However, an EPDS score at ≥ 16 gave a higher PPV (82%) and NPV (86%). An EPDS score of ≥16 for Tongan women gave the highest PPV and NPV whereas an EPDS score ≥17 for Samoan women resulted in an optimal PPV and NPV combination. Figure 1: Receiver operator characteristics by ethnicity Samoan Tongan Table 2. Sensitivity and specificity for depression assigned by different cut-off levels for Edinburgh Postnatal Depression Scale (EPDS) compared to diagnosed depressive disorder by ethnicity EPDS Samoan Tongan Area under ROC curve C = 0.8948 C = 0.8308 Cut-off Sens Spec % Correct Sens Spec % Correct 7 8 9 10 11 12 13 14 15 16 17 18 19 20 93% 93% 93% 87% 80% 73% 67% 60% 47% 33% 27% 27% 20% 20% 44% 49% 64% 73% 80% 84% 86% 90% 93% 97% 99% 100% 100% 100% 56% 69% 76% 81% 84% 84% 86% 87% 88% 87% 87% 87% 86% 86% 79% 79% 79% 71% 57% 57% 57% 50% 50% 43% 43% 36% 29% 21% 45% 55% 65% 79% 85% 87% 92% 96% 96% 99% 99% 99% 100% 100% 59% 67% 79% 82% 82% 86% 89% 88% 91% 89% 89% 89% 88% 87% Discussion Previous validation studies had not found any issues with translating the EPDS into many languages and that was also our experience. The women found the EPDS in English and the translated versions easy to complete, confirmed by the Kappa Statistic of 0.85, which showed agreement between the English and Samoan or Tongan versions. A significantly higher number of Tongan women preferred to complete the questionnaire in their language compared to Samoan women (p<0.003) which may mean that more Tongan women were more recent migrants. This study found the EPDS had strong internal consistency by language and by ethnicity. It had greater consistency in those using the English version (alpha 0.85) than the Samoan version (alpha 0.75) and Tongan version (alpha 0.81). Questions 5, 8 and 9 had poor internal consistency in the Samoan EPDS and it was these three items that reduced the overall consistency of the Samoan version. The level of consistency however compares favourably with other validation studies with high Cronbach alphas,23,24 and is similar to the alphas found in the PIF study for Samoan and Tongan women.1 The overall serious PND prevalence rate by CIDI was 17.1% and there was no statistically significant difference between the rates of Samoan (19%) and Tongan (13%) women. The serious PND prevalence rate of 17.1% found in our study was similar to the 16.4% prevalence found in the PIF study.1 The difference in PND prevalence between Samoan and Tongan in the PIF study1 cannot however be explained by our findings. Methodological differences such as the administration of the EPDS by interview and by different interviewers in the PIF study1 may not give a full explanation. The significantly higher number of Tongan women preferring the Tongan questionnaire suggests a higher number of them may be recent migrants and lower acculturation rates have been associated with a higher rate of depression.25 The difference in the EPDS cut-off points for depression between Samoan (≥11) and Tongan (≥10) women suggests that more Tongan women will have positive EPDS screening for depression given they have a lower EPDS cut-off than Samoan. Whether this difference will make a difference in PND depression prevalence between the Samoan and Tongan women will be difficult to say. The variation in cut-off scores between the different validation studies may be a reflection of the differing ethnic populations and sample characteristics of the various studied groups. In our study, the best cut-off points for the scale were ≥10 (71%, 79%) for Tongan and ≥11 (80%, 80%) for Samoan women that gave the best sensitivity and specificity for those with postnatal depression. However, using these cut-off points gave a poor positive predictive value (PPV) and this was also found in other studies.26 Differences in PPV differ with the prevalence of depression which can vary between and within populations. Populations or groups with high rates of PND have better PPVs with the EPDS than those with lower prevalence of PND. We have taken into account the prevalence rate of all PND and using the best positive and negative predictive values, alternative EPDS cut-off scores were determined. We found that the best cut-off scores were ≥16 for the English and Tongan versions (PPV 82% and 88%; NPV 86% and 88% respectively) whilst the Samoan version was ≥17 (PPV 84%, NPV 83%) for any depressive disorder. A few validation studies have differentiated an EPDS score cut-off based solely on the sensitivity and specificity of the test or solely based on its PPV and NPV. In a systematic review of 37 validation studies, a cut-off score on the EPDS at 12\/13 yielded a PPV of 17-100%.12 We feel that the two cut-off scores can be used simultaneously with the first being used for prevalence studies and the latter being used for clinical screening.26 The higher EPDS cut-off score ensures the screening test has the best performance by identifying most cases so that a diagnostic or\/and therapeutic intervention could be offered. The limitations of our study include the differing time points of administering the EPDS which was between 4 to 7 weeks postpartum and prevalence of PND by EPDS has been shown to change at different time points.27 The gap of up to 4 weeks between the EPDS completion and CIDI interview could have been shorter but the prevalence of diagnosed depression by CIDI should not differ as the instrument was designed to ascertain an episode of depression over a stated period. The NZ Ministry of Health (MOH) has decided against a formal screening programme for PND following a decision of the British National Health Service citing evidence that the condition does not satisfy screening criteria.28 The MOH has decided instead to adopt the National Institute for Health and Clinical Excellence (NICE) guideline's advice for the use of more focussed questioning in primary care, in the form of three questions.29 The routine use of the first three questions of the Patient Health Questionnaire (PHQ-3) has been promoted during the Well Child\/Tamariki Ora programme,30 even though this tool had not been validated for PND screening in NZ and has had limited validation elsewhere. Despite the recent recommendations by the NZ MOH, we propose that the EPDS is a valid and reliable tool for PND screening in Samoan and Tongan women and that its use should be continued in both primary and secondary care settings. Our findings suggest a cut-off score of ≥10 for Tongan and ≥11 for Samoan women was appropriate for screening whereas a cut-off of ≥16 for Tongan and ≥17 for Samoan was more appropriate where predictive value was important.