Abstract

Background: Since 2010, the New Zealand Defence Force (NZDF) have used post-deployment psychological screens with personnel returning from operational deployments to predict and support psychological ill-health in returning peacekeepers.

Aim: The objective of this article is to discuss the critical implications of low return rates in follow-up psychological health data in returning peacekeepers. Due to low response rates at the 4-6 month follow up screen, longitudinal analysis of mental health could not be conducted.

Methods: Two sets of responses were analysed using logistic regression from NZDF Post-Deployment screens with personnel who had served in Timor Leste and Afghanistan over the period 2010–2011. The total sample consisted of 695 cases.

Results: This study demonstrates that peacekeeping personnel with post-traumatic stress disorder (PTSD) scores above the cut-off at the initial screen returned the follow-up (FUP) screen to a significantly lower degree than their peers.

Conclusions: The results of this study suggest that among those who did not complete the FUP screen, there may be an over-representation of personnel with PTSD symptoms. If healthier subjects are more likely to return the FUP screen, post-deployment well-being may be skewed towards more positive mental health than is accurate, leading to an underestimation of the mental health burden for returning peacekeepers.

Keywords: Returning soldiers; Post-deployment psychological well-being; Post-deployment psychological screens; Mental health burden; International peacekeeping.

Conflict of interest: The authors declare no conflict of interest.

Introduction

In this article we analyse New Zealand Defence Force (NZDF) post-deployment well-being and psychological health data, collected from Service personnel returning from NZDF operational deployments in Afghanistan and Timor Leste during 2010 and 2011. Our initial aim was to enhance the NZDF’s existing Psychological Deployment Support Programme (PDSP) by determining if differences in mental health over time existed amongst sub-sets of the deployment population. However, this aim was not achieved due to low response rates by NZDF Service personnel at the 4-6 month follow-up mark. This article will discuss the critical implications of low return rates of follow up psychological health data after deployment.  Research suggests   that those leaving Military Service following operational deployment may in fact be those who are struggling the most with psychological post-deployment issues.1, 2 Low responses at the 4-6 month follow up of psychological health data as seen in this study may therefore indicate that the mental health burden of peacekeepers is underestimated.

Ethics approval and funding

This project was ethically approved by the Assistant Chief of Personnel in the NZDF and the University of Otago Ethics Committee (ref.  nr.  11/055). The NZDF had previously approved the collection of post- deployment survey data.

Methods

The data analysed in this study consisted of two sets of responses from the NZDF Post-Deployment screens over the period 2010–2011.  The first screen, the Initial Return To New Zealand (RTNZ) screen, is administered to   personnel   in   location or in the transition phase on exiting the operating environment. For this study, personnel deployed to Timor Leste completed their Initial RTNZ Screen in Timor Leste, and for those deployed to Afghanistan, they completed their screen during transition from Afghanistan in the United Arab Emirates (UAE).  At the 4-6 month mark of personnel having returned to New Zealand, they are administered a second screen through the mail, the Follow Up Psychological (FUP) screen. The data set consisted of 1,000 personnel from two deployments   within   Afghanistan   and one in Timor Leste; of these, all but a handful had completed the first post-deployment screen (Initial RTNZ), and 344 had also completed the FUPS. The matched survey data was received from the NZDF on January   19, 2012.

After initial analyses by the lead author, a statistician* from the University of Otago’s Department of Mathematics and  Statistics  was  contracted  to explore the possibilities  for  analysing  a  data  set with high numbers of missing data. A range of different methods were used† to investigate whether more information could be extracted from  the  data set despite the high proportion of missing  data  – most importantly, in identifying factors predicting psychological ill-health at the time of FUP screen completion – however, these advanced statistical techniques could not capture more information and did not change any of the conclusions. Therefore, simple descriptive statistics and logistic regression was used owing to their simplicity and ease of interpretation and understanding.

In the following sections, the findings and challenges of the data are reported and discussed. Some suggestions are made for addressing issues associated with the high percentage of missing data within screens and at follow up.  The article will finish by identifying some pertinent areas for future research on the psychological well-being of peacekeepers.

Results

As mentioned, because of the large proportion of missing data from the FUP screen, a comprehensive analysis of matched data between the Initial RTNZ and FUP screen results was not possible. Therefore, the analyses reported here are a review of sub-sets of the sample population and the probability of non- responding to the FUP screen. This information is presented in order to begin thinking on the critical issue of how to develop screening procedures that may increase the response rate for the FUP screen. As mentioned, recent research suggests that the initial post-deployment screen may underestimate the mental health burden of returning soldiers, 2 which emphasises the significance of obtaining follow up data.

The original data set consisted of 1000 personnel. Of these, 980 personnel   submitted   the   Initial RTNZ screen. However, nearly one third of these personnel did not fully complete the screen, which creates a selection bias in the Initial RTNZ results. This selection bias impacts upon the ability to gain information from the data, as it is unknown why approximately 30% of respondents chose not to answer certain items in the first screen, the Initial RTNZ. Those who had not completed the entire Initial RTNZ screen were eliminated from the study, in order to create a clean data set of complete case data.

In relation to the FUP screen, individual responses were included in the data set if any of the four scales included in the FUPs (PC-PTSD, K-10, AD and AUDIT)‡ had been  completed.  Two rotations of personnel, one from Afghanistan and one from Timor (N=204), were excluded, as insufficient time had passed since their RTNZ, and they had not yet completed the FUP screen. Subjects without complete PC-PTSD, K10, or AD scores (scales included in both the Initial RTNZ screen and the FUP screen), as well as subjects with incomplete demographic information on the FUP screen were also excluded (N=106). This elimination of incomplete cases left a sample of 695 for inclusion in the analysis.

Table 1. Complete case NZDF Post-deployment screen data, September 2013.

* Scoring above the prescribed cut-offs in any of the three psychological health measures: PC-PTSD, K-10, (A) AD.

FUP screen response rates were very low; only 1/3 (approx. 35%) of deployed personnel responded. Of the 980 who would have been sent the FUP screen for completion, only 334 had returned a completed screen. This is in contrast to the high response rate of the Initial RTNZ. This disparity in response rates is likely to be related to the administration processes associated with each screen. For those included in this data set, a NZDF psychologist provided them the Initial RTNZ screen on completion of a group psycho- education session on common transition issues. As mentioned above, the completion of this screen was conducted either in their deployment location (Timor Leste) or in a third location (UAE), prior to their home coming, and is a requirement of the NZDF post-deployment administrative procedures. On completion of the Initial RTNZ, the personnel return it to the administering psychologist. In contrast, the FUP screen is distributed by mail four–six months later, with an information sheet and a return- addressed envelope. At this time face-to-face contact with a psychologist was not required, and some of the cohort may have left the NZDF by then and/or moved from their last known   address.

Table 1 shows the complete case data used in the analyses.  The   values   for   PC-PTSD,   K-10,   (A)AD, or scoring above the prescribed cut-offs on any (marked as ‘Any’) of these screens, are results taken from the Initial RTNZ screen  using  the  NZDF  cut- offs or those recommended from the literature§. The ‘Follow-up’  column  demonstrates  what percentage of the eligible 334 personnel  completed  the follow- up screen by category, but not  what  their  scores were on  the  Follow  Up  Screen.  This information was not included due to the low response rate in comparison to the Initial RTNZ, and the subsequent risk of possibly misinterpreting the results when compared to the Initial RTNZ. For example, Table 1 shows that 35.5% of men from the original sample also completed the FUP  screen,  compared  to 56.1% of women; however due to the small number of women present in the sample (66 persons), this does not enhance the total FUPs response rate.  By the same token, we can see in Table 1 that almost twice as many NZ Europeans/Pakeha completed the FUP screen (45.9%) in comparison to Maori (27.5%).

The column ‘Reg’ displays the results of logistic regression analysis (glm in R**) which was used to estimate the effect of demographic characteristics and post deployment screens on the FUP screen response rate. ††  Results from this analysis are presented as percentage point departures from the baseline score, i.e. the change  in  the  response  rate as a percentage, if the responses to the variable in question have altered between response times (Initial RTNZ  and  FUP screen).

Based on this analysis methodology, the baseline probability for returning a completed FUP screen was 21.6%. This rate is substantially less than the mean response rate, and applies to a hypothetical baseline profile‡‡: a male, NZ European aged between 1619, a Private in the NZ Army, employed as a Regular Force Service member, who is single with no dependents, on his first deployment, which is Afghanistan§§.

This profile was used as the standardised norm against which to compare other variables. For other categories of interest to the NZDF, the following differences can be noted:

Gender

Women were more likely than men to complete a FUP screen, by 15.2 percentage points.

Ethnicity

Maori personnel were less likely to complete the FUP screen by 11.1 percentage points compared to the baseline profile. There is insufficient data to estimate an effect for other   ethnicities.

Age, Relationship Status & Family

Older personnel were more likely to return a FUP screen. Subjects aged 50 and over returned a completed FUP screen an estimated 61 percentage points over the baseline profile. However, this estimate is based on only 26 subjects, and conclusions are therefore somewhat imprecise. All respondents over the age of 25 had an estimated response rate at least 25 percentage points over the baseline profile, except those who were 3539.  In summary, those aged   20 – 35, and 40 or older were more likely to return a completed FUP screen.

Married Service personnel were more likely to return a completed FUP screen by 14.5 percentage points compared to the baseline profile. No significant difference was found between Service personnel based upon the number of dependents.

Rank, Service, and Mission

Rank had different effects for Other Ranks and

Officers. *** In comparison to the baseline profile, JNCOs (12.7 percentage points) and SNCOs (38 percentage points) were more likely to return a FUP screen. The sample size for Warrant Officers was too small to create definitive outcomes, and although no significant effect was found for this rank group, the estimated effect size was positive. For officers, personnel within the ranks of 2Lt to Capt. returned a completed FUP screen 31.5 percentage points more often than the baseline profile, but no significant effect was found for those in the rank bracket between Maj and Col.

Personnel from the Royal New Zealand Air Force (RNZAF) were 40.6 percentage points more likely to return a FUP screen, but again this result is not definitive due to the small sample size. There was not enough data on Navy personnel to conduct an analysis on this group. The missions Afghanistan III (15.1 pp), Afghanistan V (13.7 pp), and Timor Leste I (11.1 pp) all had significantly lower response rates of the FUP screen in comparison to the baseline profile.

There was no significant effect on completion of the FUP screen in relation to the number of   deployments.

PTSD

Personnel scoring above the cut-offs used in this analysis on the PC-PTSD screen had response   rates

14.6 percentage points lower than the baseline profile. This would suggest that among those who did not complete the FUP screen, there may be an over- representation of personnel with PTSD symptoms. However, due to the high levels of missing data, analyses concerning what factors on the PC-PTSD predict ill-health at the time of the FUP screen cannot be achieved. In statistics, this high rate of missing data is called nonignorable nonresponse, and can be explained in the following way.

Suppose that we had a complete set of FUP screen responses which were able to be matched to the Initial RTNZ results from each respondent. If we shuffled the FUP screen responses, split them randomly into three piles, and discarded two of the three, this would be equivalent to having a smaller sample the same as the original sample. The result would be less precise estimates of effect sizes, but no additional bias would be introduced. At the other extreme, if we sorted the interviews into three piles according to one of the measures of ill-health as reported on the PC-PTSD (such as avoidance), and then  discarded  the  two  most  negative  piles, then any analysis of the remaining pile will give us effect size estimates biased towards a lack of avoidance amongst the sample. If healthier subjects are more likely to return the FUP screen, then the current

NZDF sample may be skewed towards more positive mental health outcomes in the post-deployment period than is accurate. This would imply that the levels of mental health distress amongst those who have returned from deployment would be greater than that recorded by the NZDF. Any interpretation of results from the screens must carefully take this into account.

Discussion

Since 2010, the NZDF have trialled and implemented the use of post-deployment psychological screens with personnel returning from operational deployments. Little research on this data has yet been completed, and few studies have been conducted on the particular experience of New Zealand Service personnel on operational missions. Aside from psychological well-being, issues relating to adverse mental health outcomes which have occurred as a result of deployment are also likely to impact upon some individuals’  decisions  to  remain  in  or  leave the Service – the issue of retention  has  been  of critical importance within the NZDF in recent years. Therefore, considering the importance of learning more about the relationship between deployment and retention, this survey data has the potential to provide critical information in a number of areas. This NZDF initiative is therefore of great   significance.

The major finding from this study highlights the necessity to increase the response rate to the FUP screen so cases can be matched, and also to increase the response rate within each screen, ensuring all personnel complete all items within each screen. Recently, processes related to the administration of the FUP screen have been reviewed within the NZDF in an attempt to increase the response rate, and have proved to be successful in doing so. NZDF are also currently investigating the potential of running all future PDSPs with face-to-face contact with a psychologist, in order to ensure all personnel are provided with the best post-deployment psychological support  as  is possible.

However, the issue of low response is critical: mental health problems after deployment lead to severe difficulties such as decreased quality of life, psychosocial issues – often with dire consequences for families – or substance abuse. Further, mental health problems post-deployment have been found to be significantly associated with attrition from the military3 and initial post-deployment screening may underestimate the mental health burden of returning soldiers2. Without more information, it is impossible to determine if those who do not complete the FUP screens are experiencing greater rates of post- deployment difficulties and mental health issues. As mentioned, recent studies call for more longitudinal data to determine the long-term implications of deployment4; without matching data we cannot know the long term effects of operational deployment for the personnel involved. Enhanced response rates would allow future research to investigate differences within the population and thus allow for targeted interventions and psychological support. Enhancement  of  the  data  set  could  be  achieved in several ways, for example by emphasising the importance of answering all questions in the written information sheet attached to the screen, including verbal instruction provided by the administering psychologist and the completion of missing items during the face-to-face feedback with the NZDF psychologist, and a careful review on the layout and format of the screens in an attempt to reduce the likelihood of individuals ‘not seeing’ specific items.

Additional findings from this study relate to the profile of the non-respondent, and provide initial information on potential areas of risk   for the NZDF. However, due to the issue associated with missing data, these findings are inconclusive and further research with a fuller data set is required to substantiate these results. Nevertheless, preliminary results from this study show that the possible areas of risk for psychological  ill-health  for  members  of the NZDF during the post-deployment period are associated with those who match the following criteria; male, Maori, aged under 20 or between 35– 39, and single. Of interest, in the current study the number of deployments was not significant, although it is hypothesized that this would be a significant variable with regard to mental health outcomes should the data be complete.5

More substantially, a recent NZDF study demonstrated that Maori, to a greater extent than non-Maori, leave the NZDF after their first deployment. The reasons for this phenomenon are as yet unknown, 6 but may be correlated with the results from the present study, which highlighted that Maori are less likely to complete the FUP screen because they were more likely to leave the organization following completion of a deployment. However, another possible reason could be higher levels of psychological distress, 2 and   less access to psychological support services in the post- deployment period. Such distress may impact upon the decision to leave the NZDF following deployment.

One pertinent question is whether returning Maori Service personnel suffer from greater psychological distress and mental health problems post-deployment than non-Maori, as has been observed in the wider   New   Zealand   population.7 In order to investigate this issue, we suggest that future research reviews the post-deployment data specifically in relation to ethnicity, mental health outcomes and post-deployment retention. This information would provide the NZDF with critical information regarding Maori and retention, and provide additional information on missing data by addressing the question of whether  those  who  do not respond to the FUP screen are predominantly personnel who have discharged from Service. Addressing the proportionately low response of Maori to the FUP screen may also highlight specific needs of this group during the post-deployment period. Such research would need to examine quantitative results, but also utilise Kaupapa Maori research††† approaches in order to deepen the understanding of what specific issues Maori face post-deployment and how this impacts upon retention decisions.

Conclusion

In the present study, follow up response rates from returned NZDF personnel were too low   to allow full analysis. However, the results of this study suggest that among those who did not complete the FUP screen, there may be an over-representation of personnel with PTSD symptoms. If healthier subjects are more likely to return the FUP screen, post-deployment well-being may be skewed towards more positive mental health than is accurate, leading to an underestimation of the mental health burden for returning peacekeepers. Recent research has demonstrated that initial post-deployment screens may   underestimate   psychological   ill- health in returning peacekeepers. Hence, the major recommendation of this study is to urgently find ways to increase response rates in post-deployment psychological screens within the NZDF. Such an initiative can potentially both decrease attrition and support those struggling with traumatic experiences after peacekeeping – but who may currently be falling off the radar.

* Peter Green, co-author.

† Such as Lasso and Elastic Net.

‡ PC-PTSD: Primary Care Post-Traumatic Stress Disorder screen; K10: the Kessler Psychological Distress Scale; AD: Adjustment Difficulties screen; AUDIT: Alcohol Use Disorders Identification Test. 

§ The cut-offs used for analysis in this study were PC-PTSD: +1 (from literature), K10: 16+, and AD: 3+ (NZDF cut-offs). With the NZDF cut-off for PC-PTSD, only 1% were above cut-off – too few to get a good estimate of the effect on follow-up. With the standard, lower cut-off (used in other research) we get a significant predictor for nonresponse at follow-up. It may be worthwhile for the NZDF to lower the cut-off to the standard level in order to help identify those struggling with post- deployment.

** Generalized Linear Models in ’R’ – a programming language for statistical computing. †† For these analyses, the “Relationship Status Now” variable was removed from the analysis to avoid multicolinearity.‡‡ In line with praxis, the hypothetical baseline profile was chosen on the basis of the largest category; however, for ordered categories such as age, one end of the scale (e.g. youngest grouping) is chosen if the numbers are very similar. §§ The names of the NZDF rotations to Afghanistan and Timor are classified and therefore referred to simply as Afghanistan I, II, III, etc., and Timor Leste I and II.

*** The rank abbreviations are as follows: JNCO (junior non-commissioned officer), SNCO (senior non-commissioned officer), 2Lt (second lieutenant, junior commissioned officer), Capt. (Captain, commissioned officer), Maj (Major, commissioned officer), Col (colonel, senior commissioned officer).

††† Kaupapa Maori research is a framework or methodology for thinking about, approaching and conducting research, guided by the Maori philosophy and view of the world.