Rodney Pope, Robin Orr
ABSTRACT
Objective: To determine incidence rates of reported work health and safety (WHS) incidents and injuries in Army Reserve (ARES) and Australian Regular Army (ARA) personnel and assess the relative performance of the WHS incident reporting system, compared to ‘point-of-care’ systems.
Methods: WHS incident data for a 24-month period were extracted from a military database. Reported WHS incident and injury rates for both populations were calculated and compared. The WHS injury rates were compared with previously published injury incidence rates based on ‘point-of-care’ incident reporting in Army populations to ascertain relative performance of WHS and ‘point-of-care’ systems.
Results: In both populations combined, 15065 incidences (11263 injuries) were reported. The injury rates for ARES and ARA were, respectively, 31 and 17 injuries, per 100 person-years of active service. Published Army injury reports based on point-of-care injury reporting have cited much higher soldier injury incidence rates.
Conclusion: ARES reported WHS incidents and injuries were higher than those of ARA personnel. There appears to be substantial under-reporting of WHS injury incidents on the military WHS database when compared to point-of-care incident reporting.
Key Words:
Injury, Surveillance, Military, Health and Safety, Defence
INTRODUCTION
Reserve soldiers constitute a substantial and integral part of contemporary military forces and, just like their full-time counterparts, their capabilities can be rapidly degraded by work health and safety (WHS) incidents and associated injuries. Despite these facts, rates and sources of WHS incidents and injuries are rarely reported for reservists, and this knowledge deficit limits the information commanders have at their disposal when seeking to manage associated risks. These risks affect not only the individual, and potentially their civilian workplace, but also the military teams in which they operate and operational capability.
In order to begin to address this knowledge deficit, in a recent study 1 we examined WHS incident and injury rates and patterns in Australian Army Reserve (ARES) soldiers and compared them to those in full-time soldiers in the Australian Regular Army (ARA). We found that, per capita, ARES soldiers reported fewer WHS incidents and injuries than their ARA counterparts in a recent two-year period, and we identified some key sources of injuries in both ARES and ARA populations. However, we also noted that the per capita incidence rates calculated in that study did not take into account the fewer annual days of active service typically served by ARES soldiers, for which the numbers were not available at the time of that study. We therefore recommended that future research be conducted to compare the incidence rates of WHS incidents and injuries in ARES and ARA soldiers, in terms of the numbers of incidents and injuries reported per 100 person-years (or full-time equivalent years) of active service, so that the relative level of exposure to military service was taken into account.
To date, only one other identified publication 2 has compared the reported injury incidence rates for military reserve and full-time personnel. That publication, the Australian Defence Force (ADF) Health Status report published in 2000 2, noted that ADF reserve personnel reported more than 3 times the rate of injuries reported by their full-time ADF counterparts for each full-time equivalent year of active service. While the full-time personnel reported 9 injury or illness incidents for every 100 full-time years of active service, the reserve personnel reported 29 such incidents for every 100 full-time equivalent years of active service. This was notably quite different to the per capita rate of 4 incidents for every 100 reserve personnel first presented in the Defence Health Status report 2, which initially suggested that full-time personnel suffered a higher rate of injuries and illness when the much lower annual days of active service typical of reserve personnel were not taken into account.
One difficulty in ascertaining both per capita incidence rates and incidence rates that take into account the level of exposure is the often unknown threshold for reporting of WHS incidents. In other words, what proportion of injuries thatoccur are actually reported? It may be that only certain injuries are reported (e.g. a fracture as opposed to a blister) or that only some people routinely report their injuries. When presented with comparative rates of reported injuries for different cohorts, the concern is always therefore whether any differences in reported rates represent real differences between the cohorts in actual injury rates or whether the differences are simply an artefact of different reporting thresholds in the cohorts being compared.
In addition, thresholds for reporting of WHS incidents and injuries are important, as if the threshold is too high and injuries are rarely reported, the volume and quality of data available to guide injury risk management efforts are markedly reduced. Furthermore, injuries sustained may appear to be minimal whereas in fact injury rates could be markedly higher. This data deficit impacts negatively on the statistical power of any analysis of the data to identify emerging risks or spikes in injury rates in a timely manner, with flow-on effects to command capacity to manage the associated risks and thereby maintain Army capability. If ‘near misses’, ‘dangerous occurrences’ and ‘minor injuries’ are not routinely reported, then new or emerging hazards and sources of injury risk can also be easily missed, with similar flow-on effects. WHS incident and injury reporting rates therefore constitute a key indicator of WHS incident reporting system utility for commanders. Other indicators of utility include 3: having efficient, routine and multi-purpose incident reporting mechanisms; ensuring the system has adequate and suitably tailored and timely information outputs; system capability for timely detection and command alerts regarding emerging incident trends of importance; and ensuring there is a robust feedback loop to those reporting and entering data in order to maintain their commitment to ensuring data integrity.
On this basis, the aim of this study, which drew in part on the same data set used in our other recent paper on this topic and comprised an extension to that previous study, was two-fold: (a) to determine the recent incidence rates of reported work health and safety (WHS) incidents and injuries in ARES and ARA personnel; and (b) to assess the performance of the Australian Department of Defence WHSCAR system relative to ‘point-of-care’ (health care consultation) injury incident reporting systems, with regard to injury incident capture rates.
METHODS
Research design
A retrospective cohort study was conducted to ascertain and compare the incidence rates of both WHS incidents and injuries for the complete ARES and ARA populations in the period 1 July 2012 to 30 June 2014, inclusive. The injury incidence rates derived from the WHS data sources used in this study were subsequently compared to injury incidence rates derived from DEFCARE, the predecessor WHS incident reporting system of the ADF, as well as injury incidence rates from previously published Army injury reports which used ‘point-of-care’ data capture (data capture at the time of presentation for health care), to assess differences in injury incident capture rates.
Ethics approval
Ethics approval for the study was granted by the Australian Defence Human Research Ethics Committee (ADHREC; protocol LERP 14-024) and the Bond University Human Research Ethics Committee (BUHREC; protocol RO1907). Authorisation to conduct the project was also obtained from the Australian Department of Defence and authorisation to release this paper from Joint Health Command.
WHS incident and injury definitions
For the purposes of this study, the definition of WHS incidents included all incidents recorded on the WHSCAR database for the population and period of interest, comprising: (a) minor personal injuries; (b) serious personal injuries (or illness); (c) dangerous occurrences; (d) fatalities; (e) incidents involving exposure to a hazardous substance or material; and (f) ‘near misses’. The definition of injury included only the following types of incident reported on the WHSCAR database: (a) minor personal injuries; and (b) serious personal injuries (or illness).
Data sources
WHS incident and injury data and population data for both ARES and ARA were obtained for the period 01 July 2012 to 30 June 2014. The WHS incident and injury data were extracted and provided in a non-identifiable form by an administrator of the Workplace Health, Safety, Compensation and Reporting (WHSCAR) database of the Australian Department of Defence.
The WHSCAR database is designed to record all incident reports submitted in the notification and reporting of Workplace Health and Safety incidents that have occurred in the Department of Defence 4. The data set extracted from the WHSCAR database confirmed, for each incident record, that the affected individual’s Service was Army. It also identified their serving status (part-time or full-time) and the type of occurrence, date of incident, incident status, incident severity, nature of incident, body site affected by incident, mechanism of incident, activity at the time of incident (including specific event, e.g. field exercise, if applicable), incident description, and duty status (on or off duty) at the time of the incident. The mean population sizes for ARES and ARA, across the study period, were derived from published reports of the Department of Defence 5,6. The total number of days of active service undertaken by ARES personnel, as a cohort, in each year of the study period was provided by administrators of the Army’s personnel databases and reflected actual days worked. Finally, injury incidence rates previously reported 2,7-9 for Army populations in DEFCARE, the WHS incident reporting system that predated WHSCAR, and based on ‘point-of-care’ systems of data capture were compiled to provide comparison rates for reference in evaluating the performance of the WHSCAR system, with regard to incident capture rates.
Participants and eligibility criteria
All records of WHS incidents and injuries extracted from the WHSCAR database in accordance with pre-specified criteria were checked to confirm they met the key eligibility criteria for inclusion in the study data set. Records were included in the study data set if they related to: (a) Australian Army Reserve (ARES) or Australian Regular Army (ARA) personnel; (b) an incident or injury that occurred while the person was ‘on duty’; and (c) an incident or injury that occurred between 01 July 2012 and 30 June 2014, inclusive. Records were excluded if they: (a) related to personnel from Australian military services other than the Australian Army; or (b) related to personnel from a foreign defence service, on secondment.
All WHS incident and injury records were categorised by cohort, each defined by service type (ARES or ARA) of the respective participant. These ARES and ARA cohorts formed the primary basis for subsequent comparative analyses.
Outcome measures
The primary outcome measures for the study were the incidence rates for WHS incidents and injuries that were reported as occurring in the 2-year period of interest. These incidence rates were separately calculated for each of ARES and ARA, in two forms, these being incidents or injuries per 100 personnel per year and incidents or injuries per 100 person-years of cumulative active service.
The secondary outcome measures for the study were the injury incidence rates that have been reported in several prior studies in Army populations. One of these prior studies 2 involved WHS incident data derived from DEFCARE, the WHS incident-reporting systems that pre-dated the WHSCAR database, and published in the ADF Health Status report (ADFHSR) in 1998. DEFCARE was very similar to the WHSCAR system, using almost identical data collection procedures and coding systems, and so reported incident rates from each system could be expected to be identical, if actual incident rates were equivalent. The remaining studies 7-9 used reporting of injuries at the time the injured soldier reported for health care for their injury (‘point-of-care’ injury reporting). These injury incidence rates provided a benchmark against which the stability of performance and relative performance of the WHSCAR system and its incident reporting approach could be assessed, with regard to capture rates of work-related injury incidents. The injury definitions used in all of these systems were very similar – any musculo-skeletal and other soft-tissue injuries that were serious enough to require a health care consultation met the threshold for reporting in each system. Although the time periods covered in these previous studies varied, injury incident rates reported for the US Army 9 reflected injuries reported in the year 2014, overlapping with the period of time from which the WHSCAR data was drawn. Additionally, comparison of injury incidence rates derived from the WHSCAR and predecessor DEFCARE 2 systems (Figure 1; note in this figure the DEFCARE data is represented by the ADFHSR, 1998) revealed relative stability between 1998 and 2014 in injury incident reporting rates derived from these consecutive WHS systems. These facts justify the approach employed in this study of comparing the WHSCAR injury incidence rates from the period 2012 to 2014 with rates from the ‘point-of-care’ systems in the same period 9 and preceding years that were later than 1998 7,8.
Data Analysis
The WHSCAR data were provided in a raw, non-identifiable format, in a Microsoft Excel spread sheet. Prior to analysis, the data were manually cleaned to ensure that only records consistent with the inclusion and exclusion criteria were retained. In addition, each line of data was reviewed and compared to other lines of data to ensure identification and removal of duplicate entries (same record entered twice).
WHS incident and injury data were subsequently entered into the Statistical Package for the Social Sciences (SPSS) Version 21.0 for statistical analysis. Descriptive analyses were first conducted to establish the numbers of WHS incidents and injuries that were reported in each of the ARES and ARA populations in each period, 1 July 2012 to 30 June 2013 and 1 July 2103 to 30 Jun 2014. In addition, the mean annual numbers of WHS incidents and injuries were calculated across the full 2-year period. These mean annual numbers of incidents or injuries were then divided by the mean numbers of personnel employed in the respective service type (ARES or ARA) across the 2-year study period and the resulting figure then multiplied by 100 to derive mean annual incidence rates for both WHS incidents and injuries occurring in the ARES and ARA populations, reported in terms of incidents or injuries per 100 personnel per year. Additionally, the total numbers of injuries and WHS incidents that were reported across the 2-year study period were each in turn divided by the total number of years of active service provided to the Army by each cohort (ARES and ARA), respectively, across the two-year study period, to derive incidence rates reported in terms of incidents or injuries per 100 person-years of active service (ie full-time equivalent years). When calculating total years of active service (i.e total full-time equivalent years of service) for the ARES, 232 days of active service were assumed to equate to one full year of active service (or one full-time equivalent year of service) based on the following calculation: Total days of active service typically completed in a full-time year of army service = 365d in a year – 104d weekends (or ‘in lieu’ non-service days) – 20d annual leave – 9d public holidays
Population estimates of the ARES:ARA incidence rate ratios (IRR) for both WHS incidents and injuries, indicating the ratios of incidence rates in ARES compared to ARA, were calculated using the following formula 10:
IRR = (ARES incidence rate) / (ARA incidence rate)
In these IRR calculations, the incidence rates used were those based on total number of full-time equivalent years of active service (rather than total number of personnel). The ninety-five percent confidence interval (95% CI) around the population estimate of each IRR was then calculated as 10:
95% CI = exp (ln[IRR] – 1.96 x SE(ln[IRR])) to exp (ln[IRR] + 1.96 x SE(ln[IRR]))
where SE(ln[IRR]) = √(1/[incident rateARES] + 1/[incident rateARA] – 1/nARES -1/nARA )
Finally, the injury incidence rates calculated in the current study based on data from the WHSCAR database were charted, as planned, against injury incidence rates reported in previous studies of injuries reported in Army populations via DEFCARE 2 and ‘point-of-care’ injury reporting systems 7-9. Where necessary, these previously-reported injury incidence rates were converted to provide the number of injuries per 100 person-years of active service, with reference to the authors of the respective study and to clarify details if needed, enabling a ready comparison to the incidence rates reported in the current study. The comparative chart was designed to provide an indication of the stability and relative performance of the Australian Department of Defence WHSCAR system and its predecessor DEFCARE system, with regard to injury incident capture rates.
RESULTS
ARES and ARA populations and full-time equivalent years of service
The ARES and ARA populations 5,6 and estimated total person-years of active service (full-time equivalent years of service) during the study period 01 July 2012 to 30 June 2014 are detailed in Table 1.
[Table 1 available in download]
Reported work health & safety (WHS) incidents
A total of 15065 WHS incidents were reported across the two-year period of the study (2012-2013, n=7633; 2013-2014, n=7432; Table 3). Table 3 provides the incidence rates for reported WHS incidents calculated for each Service Type and for Army as a whole, based on the figures from Table 2. IRR are also provided in Table 3, indicating the ARES: ARA ratio of incidence rates for WHS incidents.
[Table 2 and 3 available in download]
Reported Injuries
A total of 11263 injuries (comprised of minor personal injuries and serious injuries) were reported across the two-year period of the study. Table 4 details the numbers of injuries reported in ARES and ARA populations in this period. Table 5 provides the incidence rates for reported injuries calculated for each Service Type and for Army as a whole, based on the figures from Table 4. IRR are also provided in Table 5, indicating the ARES:ARA ratio of injury incidence rates. The figures presented in Table 5 indicate that the reported injury incidence rate was stable in ARES and in ARA populations, year-to-year.
[Table 4 and 5 available in download]
Comparison of WHSCAR to previously published Army injury incidence rates
Figure 1 provides a comparison between injury incidence rates calculated for ARES and ARA populations in the current study, based on 2 years of WHS incident records contained in the WHSCAR database, and benchmark injury incidence rates derived from previously published reports of injuries in various Army contexts 2,7,8.
[Figure 1 available in download]
Three 7-9 of the four previously published reports 2,7-9 indicated much higher injury incident rates for Army personnel than the rates calculated in the current study based on records from the WHSCAR system. The US Army injury incidence rate 9 of 160 injuries per 100 person-years of active service is indicative of a ‘whole-of Army’ injury incidence rate, and this injury incidence rate lies midway between incidence rates reported for Army recruits and for an operational brigade in the Australian Army, and thus is probably a sound estimate of overall actual injury incidence rates for Army populations, when considering injuries requiring a health care consultation. The injury incidence rates derived in this study from the WHSCAR database are similar to those derived from its predecessor DEFCARE system 2 and represent only 11-19% of the above estimate of the true incidence rate for injuries that are of sufficient severity to require a consultation with a healthcare provider. Injury incidence rates derived from the WHSCAR database in the current study were slightly higher than those derived from its predecessor, DEFCARE 2, but the relative similarity of these rates when compared to the ‘point-of-care’ rates (Figure 1) suggests that injury reporting rates in these WHS incident reporting systems have probably been quite stable over the sixteen-year period these studies span.
The three published reports 7-9 which reported much higher injury rates in Army personnel than the rates reported from DEFCARE or WHSCAR records all used a ‘point-of-care’ approach to injury reporting – for the Australian 3rd Brigade report 8, this fact was confirmed by discussion with one of the study authors. In the other two reports, examining Australian Army recruit injury rates 7 and US Army injury rates 9, this information was provided in the report itself 7or report source 9. In this ‘point-of-care’ approach, injuries were recorded by healthcare personnel at the time when injured personnel reported with their injuries to Army healthcare facilities. In contrast, the WHSCAR system and its predecessor DEFCARE system both used a system of reporting which depended on the injured soldier and their supervisor reporting the injury incident directly to the reporting system, in accordance with Australian Department of Defence policy 4. In most instances, this latter approach did not involve healthcare providers.
DISCUSSION
The primary aim of our study was to establish the incidence rates for reported WHS incidents and injuries sustained by Australian Army part-time (ARES) personnel during periods of active service and compare themo with rates reported by full-time personnel. In the ARES, 34 WHS incidents were reported for every 100 person-years (ie full-time equivalent years) of active service. In the ARA, 23 WHS incidents were reported for every 100 person-years of service, suggesting that ARES soldiers experience almost 50% more WHS incidents than their full-time counterparts in the ARA, when days of active service are considered. The differences in injury incidence rates were even more pronounced. In the ARES, 31 injuries were reported for every 100 person-years of active service. In the ARA, 17 injuries were reported for every 100 person-years of service, suggesting that ARES soldiers experience 80% more injuries than their full-time counterparts in the ARA when days of active service are considered.
Interestingly, however, these substantial incidence rates for both WHS incidents and injuries appear to represent just ‘the tip of the iceberg’ in both ARES and ARA populations. Comparison of the injury incidence rates alone, derived from the current study of the ARES and ARA populations, to benchmark injury incidence rates from other published studies of Army populations that have used ‘point-of-care data capture’ 7-9 revealed that the WHSCAR database interrogated in the current study is probably only capturing reports of between 11% and 19% of all injuries actually suffered by soldiers which are serious enough to warrant them seeking health care advice. This means that approximately 80-90% of all injuries suffered by ARES and ARA soldiers that are serious enough to require health care are probably not being captured on the WHSCAR system.
This latter finding has several important implications. First, given these very substantial data deficits, it is impossible to say whether the differences in reported incidence rates for WHS incidents and injuries identified in the current study are indicative of real, underlying differences in injury risks between the ARES and ARA or simply an artefact of incomplete reporting and differences between the ARES and ARA in typical reporting thresholds for such incidents and injuries. Table 3 indicates that rates of recorded serious injuries, though 23% higher in ARES than ARA, were nevertheless much more similar between these populations than rates of recorded minor injuries, which were almost twice as high in ARES as in ARA personnel. This finding supports the notion that under-reporting of WHS incidents is one likely cause of the observed differences in recorded incident rates between these populations, since under-reporting of minor injuries is more likely than underreporting of serious injuries. Further research with more robust data capture or sources is required to elucidate this matter. Nevertheless, it should be noted that ARES soldiers are at substantial risk of being injured and a strong focus on management of injury risks not only in the ARA but also in the ARES is warranted.
Second, noting the importance discussed in the Introduction to this paper of comprehensive data capture for adequately informing management by commanders of WHS incident and injury risks and their flow-on effects to personnel availability and operational capability, it would seem important that the evident deficit in incident reporting and data capture is noted and addressed. A key lesson learned in the benchmarking exercise conducted as part of the current study is that those benchmark incident reporting systems which captured 5 to 10 times as many of the actually-occurring injuries in soldiers all employed a ‘point-of-care’ approach to reporting, in which health care personnel created a record of the incident or injury at the time when an injured soldier presented for healthcare. The WHSCAR system and its predecessor DEFCARE system do not employ this approach, and instead the soldier affected by the incident or injury and their supervisor are responsible to report the incident to the system (and notably not to a person) 4.
On this basis, it would appear prudent that developers and administrators of military WHS incident reporting systems ensure that point-of-care reporting mechanisms are incorporated in these systems to maximise data capture and so support WHS incident and injury risk management by commanders. However, it should also be noted that point-of-care reporting systems will not readily capture data on near misses, dangerous occurrences and exposures to hazards, unless they result in some sort of injury or concern requiring health care. Thus, future WHS incident reporting systems should be developed to use hybrid systems for data capture, incorporating both point-of-care and soldier/supervisor reporting approaches, with the latter approach designed to be as user-friendly as possible.
While this study has considered some aspects of WHS incident reporting systems, it should be noted that ensuring these systems can properly and comprehensively inform command risk management efforts in a timely manner depends on optimisation of many factors other than the data capture approach employed. These other factors are explicated in a previous comprehensive report by McKinnon and colleagues 3, which was based on a study conducted in the Australian military context. That report should also be considered by developers and administrators of WHS incident reporting systems and the military services they seek to serve. Of note, data capture is also very likely to be enhanced by optimising many of these other factors 3.
Finally, even when WHS incident reporting systems are optimised, their proper use by commanders to inform management of risks that these systems can identify will depend heavily on what the support commanders receive to identify and manage such risks. Where commanders and military organisations benefit most from demonstrating low rates of WHS incidents and injuries, rather than from demonstrating sound practice in risk identification and management, interest in enhancing the rates of identification of WHS incidents and injuries will be limited 11. Lower levels of reporting and thus poorer system functioning in such contexts yield perceived benefits. Determinants of a sound reporting culture are well explicated in the paper by van der Schaaf and Kanse,11 which constitutes further recommended reading for developers and administrators of WHS incident reporting systems and commanders.
CONCLUSION
This retrospective cohort study evaluated the incidence rates of reported WHS incidents and injuries that were sustained in both ARES and ARA personnel over a recent two-year period. Previously available information 2 of this nature is limited and aged. The results of the current study suggest that ARES personnel report 50% more WHS incidents and 80% more injuries than their ARA counterparts, when actual days of active service are considered. However, while the current study has used the best currently-available data set and certainly confirms substantial WHS incident and injury risks in both ARES and ARA populations, which we recommend should be a focus of risk management efforts, we have also identified highly-probable, very substantial levels of under-reporting in this data set. These high levels of under-reporting mean that we cannot be certain whether the differences in WHS incident reporting rates observed in this study represent true differences in underlying levels of risk or reflect uncertainties in the data related to substantial under-reporting of incidents.
On this basis, a further important recommendation from the current study is that developers and administrators of military WHS incident reporting systems and the command elements they serve take steps to ensure the systems they use incorporate ‘point-of-care’ reporting of injury incidents as well as continued reporting by affected personnel and supervisors of near misses, dangerous occurrences and exposures that do not result in significant injury. Additional advice regarding optimisation of WHS incident reporting systems and building a reporting culture has also been provided, based on recent research findings, and is worth considering.
Funding:
This research was supported by a grant from the Defence Health Foundation.
Conflict of Interest:
Authors Pope and Orr have received research grants from the Defence Health Foundation for research investigating injury rates in Australian Army Reserve personnel, a project from which this paper arose.
