JRSM >
Volume 95, Number 11 >
Pp. 549-551
Centre for Health Economics, University of York, UK
1 Department of Management, University of St Andrews, UK
2 Department of Health Sciences, University of York, UK
Correspondence to: Professor H T O Davies, Centre for Public Policy and Management, University of St Andrews, St Andrews KY16 9AL, UK E-mail: hd{at}st.and.ac.uk
 | INTRODUCTION |
|---|
 TOP INTRODUCTION POTENTIAL HAZARDS IN PERFORMANCE...MAXIMIZING THE POTENTIAL OF...REFERENCES |
|---|
| POTENTIAL HAZARDS IN PERFORMANCE MEASUREMENT |
|---|
|
TOPINTRODUCTIONPOTENTIAL HAZARDS IN PERFORMANCE...MAXIMIZING THE POTENTIAL OF...REFERENCES |
|---|
Tunnel vision arises when the system forces concentration on areas covered by the performance measurement scheme to the exclusion of other important but unmeasured areas. So far, for example, efforts to meet the old-style NHS efficiency targets were often directed at increasing measured activity within the hospital setting, at the expense of improvement in outcome that might have been achieved through increasing care outside the hospital. Similarly, efforts to meet waiting-list targets may have diverted attention to managing the lists (to improve ratings), rather than maximizing improvement in health by prioritizing according to clinical need. The NHS performance assessment framework seeks to redress this balance by covering six areas intended to give a balanced view of healthcare performance2. However, there is clearly some way to go: the Government's consultation document suggested 470 possible indicators, only 7% of which fall into the ‘health outcomes’ category3.
The tendency towards tunnel vision might be offset by devising a large number of measures none of which are to be neglected. However, the cost and data-collection burden would be prohibitive. Clinical staff would become over-whelmed and effort could be wasted trying to identify the indicators that really count4.
The need for a clearer focus is the rationale behind the notion of ‘headline’ indicators, published and used to hold organizations to account. Other indicators will be used for internal ‘benchmarking’ purposes only5. Key indicators should focus on priority issues; however, priorities can differ between clinicians, managers, patients and politicians, so some sort of consensus is needed.
Suboptimization occurs when NHS staff pursue narrow targets at the expense of the objectives of the system as a whole. For instance, a target set in the hospital sector, such as an increase in day-case rate or a shorter length of stay, may take no account of the increased burden for primary care or social services. Suboptimization may also occur within an organization. Multispecialty teamworking may be compromised if team members pursue targets set in their own specialty without taking account of the impact on the overall effectiveness of the team and the outcome for the patient. One reason for the apparent success of the cancer collaboration pilots in reducing waiting times may be that the whole multidisciplinary team was made responsible for meeting targets.
Suboptimal behaviour can be addressed through mechanisms to encourage partnerships and joint working (e.g. joint health and social service targets set out in the NHS Plan) and strategies to encourage and reward teamworking (e.g. devolved responsibility and team bonuses).
Myopia is the tendency to focus on short-term issues at the expense of long-term considerations that may only show up in performance indicators in several years' time. The classic example is where indicators are focused on curative rather than preventive services. Setting targets that seek to measure improvement over time, rather than on an annual basis, may help to offset myopia. The UK Government has indeed set longer-term targets (e.g. improvements in mortality rates from major killer diseases by 2010) but whether this is sufficient to promote improvements via preventive rather than curative care remains to be seen. The use of process indicators to monitor progress in moving towards long-term goals may help1—as would the fostering of long-term careers within an organization.
Complacency refers to a lack of motivation for improvement when comparative performance is deemed adequate. Staying out of the headlines by ensuring that performance is ‘good enough not to attract attention’ (even where the potential for better results exists) may be a strong incentive for many NHS staff faced with multiple demands on their time6,7. For example, if surgical performance is in the middle of the national distribution, the focus may be on outliers where performance is exceptionally poor. Complacency of this sort can be avoided by measuring performance continuously over time and linking rewards to improvement.
Measure fixation occurs when the pursuit of success as measured, rather than as intended, becomes the main focus. There has been debate about the benefits of the two-week maximum waiting time for cancer referrals. This could override clinical judgment and distort the flow of patients in favour of those who might have cancer, and in turn increase the length of time to treatment for cancer patients overall. Additionally, there is emerging evidence that in some specialties the standard is being met at the expense of an increase in the waiting times of routine referrals whilst not necessarily identifying treatable causes of cancer8. The tendency towards measure fixation may be offset by greater involvement of front-line staff in defining targets and measures and by ensuring there is an evidence base to support the use of specific targets.
Misinterpretation in the form of incorrect inferences about performance can arise where it is difficult to allow for the full range of potential influences on a measure1. Failure to take fully into account case-mix and other environmental factors in interpreting ‘league tables’ is a classic case. Better methods for adjusting clinical data can help avoid misinterpretation; at a specialty level, some professional associations have been working with statisticians and others to produce more meaningful comparative audit data9,10,11. Recognition of the limitations of indicators, by relying on them as signposts rather than as methods of control, is also important, as is the use of local knowledge to make sense of apparent variations in performance12,13.
Gaming occurs when staff seek to influence measured performance by deliberately altering variables other than clinical quality. For instance, an inadequate risk adjustment methodology might allow selection of low-risk patients (and refusal of high-risk patients) so as to secure favourable outcome figures14. If social influences on outcome are not properly accommodated, clinicians may indeed become reluctant to practise in areas with disadvantaged social circumstances. Gaming can be offset by careful attention to risk adjustment methods, and by developing indicators that can be used to test for selection bias. There may even be a need to put in place incentives to encourage the treatment of high-risk patients.
Ossification refers to the organizational paralysis that can arise from an excessively rigid system of measurement. For example, use of day-case rates as an indicator of performance in gynaecology may inhibit the adoption of new techniques that can be applied in the outpatient department. To avoid such side-effects the performance measures must be reviewed regularly; however, a balance must be struck since constant changes will be disruptive and will make long-term change difficult to track.
| MAXIMIZING THE POTENTIAL OF CLINICAL PERFORMANCE MEASUREMENT |
|---|
|
TOPINTRODUCTIONPOTENTIAL HAZARDS IN PERFORMANCE...MAXIMIZING THE POTENTIAL OF...REFERENCES |
|---|
With a few notable exceptions (e.g. intensive care17), such developments have operated at regional rather than national level and offer limited opportunities for comparison because of difficulties with risk adjustment. However, several specialties—such as vascular surgery, coloproctology, radiology and upper gastrointestinal surgery10,11—are making progress with audited outcomes. These initiatives will enable hospitals and individual clinicians to track their results prospectively against national standards with suitable adjustment for casemix, severity of illness and comorbidity. The systems can be designed in ways that avoid many of the dysfunctional consequences identified above. Professional ownership, and freedom to choose measures that focus on clinically meaningful events, will reduce the tendencies towards tunnel vision, ossification and measure fixation. Misinterpretation and the consequent gaming will be limited by the knowledge of local factors that influence outcomes and by robust methods of risk adjustment.
Attention also needs to be paid to the context within which performance measurement systems operate. There is to be a concerted effort to devolve power to front-line staff throughout the NHS, involving them in decisions at all levels, reducing hierarchies and developing self-managed teams18. If performance improvements are to be achieved, the systems that measure and monitor clinical performance will need to reflect the new structure and environment. The imposition of top-down external systems will not fit well within the new NHS: performance measurement systems will be a helpful tool to improve clinical outcomes if they are owned by professions and build upon the initiatives that have already begun. More generally, as the Bristol Enquiry emphasized, performance measurement is unlikely to be effective unless implemented within a favourable professional culture of enquiry and continuous improvement19.
| Acknowledgments |
|---|
| REFERENCES |
|---|
|
TOPINTRODUCTIONPOTENTIAL HAZARDS IN PERFORMANCE...MAXIMIZING THE POTENTIAL OF...REFERENCES |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
