Introduction
The opportunity to gain feedback from customers is an important facet of improving the delivery of health services. In the Defence Health Service this is primarily achieved through Customer Satisfaction Surveys per Health Directive 920. However this is focused on patients and misses the other customer base: the Commanding Officer (CO). This shortfall was identified during the Area Health Services – Western Australia (AHS-WA) Business Planning Workshop in September 2006. The Senior Health Officer recognised that most CO’s have certain expectations in the delivery of health services but there was no method available to ascertain what these were. This need was later highlighted by some CO’s whose expectations were poorly identified and therefore could not be met. From a health staff perspective there appears to be a general understanding of where problems exist however there was an inability to quantify them and this precluded prioritising those issues requiring action. To address this, the AHS-WA Health Systems Manager (HSM), Mr Dave Parry (DP), was tasked with developing, implementing and consolidating the results of the first AHS-WA Commanding Officer Satisfaction Survey (COSS).
Development
The focus on the development of the COSS was to create a survey tool that was:
- quick and simple,
- not Service-specific, and
- provided information that was useful in improving the delivery of health services.
One of the most important components of selecting the survey tool was to make sure that it could be received by email, completed using a box tick process and then emailed on completion. It was envisaged that the ease of use would improve the completion rate and this would provide AHS-WA with a more consolidated overview. To achieve this MS Excel® spreadsheet was chosen due to its simplicity and useability. The survey tool was developed with assistance from the Senior Physiotherapist Fleet Base West Health Centre, Mr Ashbee Robinson.
Methodology
This survey instrument is intended as a management (quality) tool and has no clinical application per se. There are no references or supporting documents regarding the creation of this tool, noting this was an identified shortfall in the feedback process. The HSM reviewed any correspondence available that provided feedback from CO’s on the delivery of health services. This assisted the HSM (DP) in identifying the key areas that CO’s were concerned about or affected their ability to maintain a healthy, fit and operationally ready unit. As a result, seven questions were developed to cover the main issues that would elicit the best feedback possible. The questions were:
1. Provision of health services
2. Keeping Heads of Departments (HOD) and Divisional Officers (DO) informed
3. Management of illness and convalescence
4. Deployment assistance
5. Keeping CO’s informed after hours
6. Timely and accurate Senior Medical Officer (SMO) advice
7. Level of Key Performance Indicators (KPI) for Individual Readiness
To measure the level of satisfaction each question had a 1 to 5 rating as follows:
1. Completely Dissatisfied
2. Somewhat Dissatisfied
3. Satisfied
4. Quite Satisfied
5. Completely Satisfied
Additional to the above there were three questions at the end of the survey that enabled the CO to make specific comments on the performance of the health facility, specifically:
1. What were they doing well?
2. What needed to be improved?
3. Are there any specific concerns or suggestions?
As this was the first AHS-WA COSS, the choice of where the survey was sent was important. As the primary role of any health facility is the provision of quality health services to optimise ADF operability, it was determined that the focus would be on operational units. The units and supporting health facility were:
Navy: Seven Major Fleet Units (Fleet Base West Health Centre) Three Submarines (Submarine and Underwater Medicine Unit – West)
Army: Nine ARA units (Medical and Dental Services Karrakatta, Regimental Aid Post (RAP) Special Airborne Service Regiment and Taylor Barracks (RAP Pilbara Regiment))
RAAF: Three Pearce-based units (Health Services Flight RAAF Pearce) Prior to initiating the survey the aim was for an overall rating for AHS-WA health facilities of 3 (Satisfied), with an 80% return rate. Results
The survey was conducted over an 8 week period (March – May 2007) that enabled deployed CO’s to respond according to operational demands. The results were valuable as they provided a clear indication on what needed to be improved and also what was being done well.
Navy Fleet Base West Health Centre (FBWHC) Overall Score: 2.49
On analysis, there was significant disparity on certain questions between the replies from ships. For example, in one question one ship rated FBWHC as 5 – Completely Satisfied, whereas two ships rated this question as 1 – Completely Dissatisfied. This had an affect on the overall rating of the Navy and could only be associated with the expectation and previous experiences of some CO’s. With the exception of keeping CO’s informed after hours, FBWHC rated below Satisfied in all other areas (Figure 1). The disparity in responses attributed to this, however, issues have been identified that can be improved and action has already been taken to address these.
Figure 1. Overall COSS rating for Fleet Base West Health Centre Submarine and Underwater Medicine Unit – West (SUMU-W) Score: 2.76
The submarine CO’s appeared reasonably satisfied. The two key areas that needed improving were the level of deployment assistance and the management of illness and convalescence (Figure 2). This was expected, as these are the two issues that have the greatest impact on the operational capability of a submarine. Although constrained by personnel issues, it has been acknowledged that these areas need improving and in consultation with the AHS strategies have been developed to correct this anomaly. Prior to the implementation of the COSS, SUMU-W made some process improvements in their management and communication of personnel who were Temporary Medically Unfit. This initiative was clearly identified in the survey as a factor that had positive outcomes and specific comments were made by CO’s to this affect.
Figure 2. Overall COSS rating for Submarine and Underwater Medicine Unit – West A
Army Medical and Dental Services Karrakatta (MDSK), RAP Special Airborne Service Regiment (SASR) and Taylor Barracks (RAP Pilbara Regiment) Score: 2.82
One of the difficult aspects of conducting this survey for Army, is that of the nine units surveyed, only two have Australian Regular Army (ARA) CO’s. The seven remaining units are commanded by General Reserve (GRES) Army personnel. As these CO’s do not perform this role on a full-time basis as do their ARA counterparts, it was decided to survey the senior non-Commissioned Officers (NCO) of the units as providing the most valuable and comprehensive feedback. The main area that was identified as needing improvement was the provision of health services (Figure 3). This was focused on MDSK as they supported eight units, of which none were geographically located on the same base. This was identified as a particular problem prior to the survey and the results further reinforced that CO’s or their representatives were not satisfied with the time lost in travelling to attend medical appointments. There is action in place to alleviate this problem and locate the medical facility at Karrakatta. The areas where MDSK rated highly were in keeping Supervisors (referred to as HODs and DOs in survey) informed, keeping CO’s informed after hours and the management of illness and convalescence. Positive comments were received on their proactive response to short-notice requests and managing Pilbara Regiment specialist appointments. This was extremely important feedback to MDSK as they had recently undergone a large staff turnaround and had only one Medical Officer (50% capacity).
Figure 3. Overall COSS rating for Medical and Dental Services Karrakatta, RAP Special Airborne Service Regiment and Taylor Barracks (RAP Pilbara Regiment)
Air Force Health Services Flight RAAF Pearce (HSFPEA) Score: 3.43
The only area that rated slightly below Satisfied was in keeping CO’s informed after hours. All other questions rated Satisfied or above with two questions receiving an overall rating of Quite Satisfied: the provision of health services and deployment assistance (Figure 4). This was an excellent outcome for HSFPEA as they had been able to maintain a high level of customer service whilst working extremely hard towards and achieving ISO 9001 certification.
Figure 4. Overall COSS rating for Health Services Flight RAAF Pearce
Discussion
Area Health Services – Western Australia (AHS-WA) Score: 2.84
As the aim was to achieve an overall rating of 3 (Satisfied), 2.84 was slightly disappointing, but as this was the first COSS it at least provided AHS-WA with a starting point for improvements (Figure 5). The response rate was 87.5% which was above expectations and this provided indirect feedback that the COSS achieved the aim of being simple and timely to complete. In most areas there were no major surprises and the wide variation in ratings between health facilities provided clear direction to the medical managers on where to focus the effort to provide CO’s with a quality health service.
Figure 5. Overall COSS rating for Area Health Services – Western Australia
Outcomes The outcomes of the survey were sent individually to the medical managers and included a summary of the positive and negative comments. This enabled the medical managers to brief their staff on the outcomes and to develop strategies to address problems capable of being achieved. As a result, the outcomes that have already been accomplished at the health facilities are:
1. Improved staff morale
2. Enhanced customer focus
3. Enhanced opportunity for improvement processes
4. Proactive approach in addressing problems
5. Quality improvement
6. Process approach
One of the most positive outcomes of the COSS process was the identification by the Major Fleet Unit CO of the need for a FBWHC Health Liaison Officer (HLO). The SMO FBWHC and his staff developed the HLO position taking into account that the primary roles were in training to address expectations and previous experiences, as well as providing a mentorship role to medical sailors on ships. The HLO position was created and filled in June 2007, and as a result the expectations and understanding of the delivery of health services has already seen positive outcomes. It is envisaged that the need for the HLO role will be reinforced in the next COSS and it is expected that there will be an improvement in FBWHC’s ratings.
The Future
Where to from here? The intent is to conduct the COSS every 12 months with the ultimate aim to achieve overall ratings in all questions of greater than 3 (Satisfied). It is suggested that the initial COSS identified the major issues that were affecting the delivery of health services and through an ongoing proactive approach should reduce the number of issues that are process focused. The prime focus of the initial COSS was on operational units, whereas the focus should now change to expanding the survey to all CO’s in WA. This will ensure that AHS-WA is receiving a true indication on the delivery of health services, as operational and non-operational commanders have different needs and expectations. Another question to be posed is – has the COSS the capability of being applied on a national basis? There is a definite need to expand our feedback process and not focus on patients alone. Inclusion of the CO’s assessment provides a broader viewpoint from a management and operational perspective. Furthermore, the use of the COSS will enable health facilities to measure their contribution to the Defence Health Service Division (DHSD) Mission to ‘Optimise the health of ADF personnel’. Priority 3 and 4 of the DHSD Mission espouse to ‘Provide quality health governance to deliver an effective and efficient healthcare mission’ and ‘Improve the quality of health care to the ADF member’ respectively, and it is the authors’ belief that the COSS provides a survey tool that links these priorities. The COSS has been forwarded to the Director Joint Health Support Agency (JHSA) to determine if this survey tool has national implications in obtaining feedback from the CO for whom JHSA provides health services. As JHSA is responsible for the provision of non-operational health support services to the ADF it is imperative that a method be identified.
Cautions
As with any method of obtaining feedback, there were some areas that were identified during this process that had the potential to undermine the ability of achieving our aim. Firstly, there were some negative comments that were not constructive or helpful. The key aspect here is to get past the emotional component and move on. As outlined previously the initial COSS uncovered those issues that are at the forefront of the commander’s mind and this may be attributable to the emotional responses that are a product from previous experiences. As we move through the emotional aspect to a more process approach, both the health facility and commander will benefit. Secondly, the process of creating a survey tool that was not complicated to use was the attributing factor to the response rate that AHS-WA achieved. In developing this survey tool, AHS-WA was cognisant of the current ADF tempo and as such wanted to promote to the commanders that it was beneficial to complete and that it was not a long, arduous chore that would not be acted upon. This was extremely important as there seems to be a general perception that surveys are conducted, data correlated, results published, but then nothing happens to address the issues identified in the survey. Finally, there is a need to ensure that surveys are not from too many sources. As the survey is focused on providing the health facility with feedback on their performance and delivery of services, the survey should be managed from the responsible Area Health Service, not through their Command and Control (C2) lines of responsibility. This is important as the C2 of the unit surveyed may not be the same as the supporting health facility. For example, a Fleet Base West based ship’s C2 is Fleet Headquarters, whereas the supporting health facility is FBWHC whose C2 is HMAS STIRLING/JHSA. The CO HMAS STIRLING and JHSA have the authority and resources to contribute to changes that are required to improve the delivery of health services, whereas Fleet Headquarters do not. From an Army perspective, there were some concerns that the COSS was cutting across command chains, however once explained that this was an AHS-WA quality initiative, there was an 80% response rate. As this was a quality initiative, AHS-WA did not seek nor require Defence Health and Human Performance Research Committee approval as required by Australian Defence Force Publication 1.2.5.3 Health and Human Performance Research in Defence – Manual for Researchers.
Conclusion
The creation of and subsequent conduct of the COSS was a quality initiative from an AHS-WA Business Planning Workshop that identified a shortfall in obtaining feedback from one of our primary customers: the CO. The aim of conducting the COSS was to achieve an overall rating of 3 (Satisfied) however AHS-WA fell just short with a 2.84 and of the 27 units to which the survey was forwarded, 24 (87.5%) responded. There were many reasons why the aim was not achieved and these can be attributed to the disparity of responses to certain questions including the difference between needs, expectations and previous health care experiences of different CO’s. As a result of the COSS, the medical managers now have invaluable feedback on the performance of their health facility that not only includes areas that they need to improve on but importantly, knowledge of what they do well. Conducting the COSS clearly identified that health facilities need to obtain as much feedback from as many of our customers as possible to improve our practices. We may perceive that we know how we are performing but if we can obtain some honest feedback on how we actually are performing, then why not use one of the best sources: the CO. There is no point in creating and conducting a survey unless there is a plan and the resources to act on the results. Many a plan fails due to the lack of resources to achieve the objectives and if the COSS is to be considered for national application and the Defence Health Services (DHS) is serious about achieving its mission, then there must be resources that enable us to achieve it. Commanders are an invaluable asset in providing DHS with direct feedback on the performance of their health facilities. DHS need to ensure that the CO’s understand that we have the capability and resources to improve our processes that will result not only in an improved level of service but importantly, health will be seen as an integral component of the commander’s ability to maintain operational readiness.