JRSM >
Volume 96, Number 7 >
Pp. 338-342
Picker Institute Europe, King's Mead House, Oxpens Road, Oxford OX1 1RX, UK
Correspondence to: Angela CoulterE-mail: angela.coulter{at}pickereurope.ac.uk
 | SUMMARY |
|---|
 TOP SUMMARY INTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Participants felt that independent monitoring of healthcare performance is necessary, but they were ambivalent about the value of performance indicators and hospital rankings. They tended to distrust government information and preferred the presentational style of 'Dr Foster', a commercial information provider, because it gave more detailed locally relevant information. Many participants felt the NHS did not offer much scope for choice of provider.
If public access to performance information is to succeed in informing referral decisions and raising quality standards, the public and general practitioners will need education on how to interpret and use the data.
| INTRODUCTION |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Little is known about the extent or nature of public demand for this type of information, or what patients think about using it to choose providers. We report a focus group study to investigate views on the existing range of performance indicators and to find out what type of information patients and members of the public want.
| METHODS |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
|
Participants for five of the six groups were recruited by an agency using street recruiters who provided their homes as venues. The carers' group met in a hospital. Participants were paid £30. A topic guide was used to focus the discussion. Participants were asked for their views on measuring and comparing performance in the NHS, what should be measured and who should be responsible for monitoring performance. The study predated the launch of the Patient Choice schemes, but groups were conducted a few weeks after publication of the Mail on Sunday's 'Good Hospital Guide' which was produced by Dr Foster. This was used together with other press cuttings and material downloaded from the Department of Health (DH) and Dr Foster websites, to stimulate discussion. Each group discussion was recorded and the transcripts were read and analysed independently by the focus group facilitator (HM) and another researcher (LD), who identified and agreed on the main themes.
| RESULTS |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Some felt that publication of hospital rankings could lead to an improvement in standards, but others highlighted the possible negative results of 'naming and shaming.' Of particular concern was the morale of staff working in poorly performing hospitals and the consequences of heavy demand on successful hospitals. Fears were expressed that publication could raise alarm, particularly amongst more vulnerable people. Several participants commented negatively about the cost of gathering performance information, saying the money could be better spent elsewhere in the NHS.
Reactions to Department of Health performance ratings
Before being shown examples of the performance ratings, most participants
had only a vague awareness of the recently published hospital league tables.
There was a good deal of confusion between the DH ratings and the Dr Foster
guides. Hardly anyone knew exactly what star rating their local hospital trust
had been awarded.
| Box 1 Effects of measuring and comparing performance 'Because I use performance indicators in my job and it is a way of seeing what people are doing, I think it is a good idea.' 'It would make whoever is at the bottom of the list pull their socks up.' 'You have got to shame them into doing something about it.' 'It can dishearten them, can't it... demoralize the staff.' 'Nurses could look at this and say, well I don't want to work there, it's not a very good hospital.' 'It might lead to overcrowding in say the better hospitals: oh we'll go there.' 'I think to target individual hospitals is sort of scaremongering really for the people who live around those areas.' 'It might make the managers manipulate the figures so they don't look so bad next time.'
|
| Box 2 Reactions to Department of Health's performance
ratings 'I just remember hearing something on the radio and I thought my God, how quick to damn a hospital which has probably got superb staff in there.' 'I get so angry with league tables because we don't know the full facts. I'm sorry but they would have to print a 50-page document for me.' 'Government figures don't mean anything to me because government is baloney.' 'I think that if the Government sent them out then maybe you've got a better chance of believing it.' 'I am not political, but I think most of them massage figures to suit themselves.' 'I don't trust this waiting list thing because I know for a fact they don't put you on the waiting list until they know you can be treated.' 'It depends how much it's costing really. If it's costing a couple of million to get it all put out, that to me is wasting money because two million can go into a kiddies' hospital.'
|
There was considerable mistrust of the DH ratings (Box 2). In many cases this stemmed from a general distrust of any Government statistics. There was an expectation of manipulation by Government and hospital managers. Several participants were particularly sceptical about the figures on waiting times and cancelled operations. Participants were confused about how the data for performance ratings were collected and assessed.
Reactions to Dr Foster information
Several participants had seen the 'Good Hospital' and 'Good
Consultant' guides produced by Dr Foster (Box 3). Participants generally
found these more user-friendly than the DH ratings, although some were
confused about how to interpret part of the data. Basic information in these
guides about hospital facilities, public transport etc. was felt to be useful
for patients and their visitors. The detailed information on waiting times in
specific clinical areas was also considered useful.
| Box 3 Reactions to Dr Foster information 'It seems to be better laid out and a little write-up on each hospital which is analysing the figures they have found out rather than just slapped down on a page.' 'Something like this [Dr Foster's Good Hospital Guide] would be good if it was delivered with your free papers.' 'This is much more useful because it is down to basic questions with basic answers, like does the hospital have a newsagent. Little things like that are the sort of things that it's handy to find out about.' 'But who pays Dr Foster then?' 'You don't know who's been the financial backer of something like this. I don't take much information as read. I'm a "why" girl, I'm afraid.' 'So if you look at it like that, it is saying that it [Walsgrave Hospital] is a good hospital... and we know it isn't.'
|
Wary of vested interests affecting the reliability of the information, some participants wanted to know who funded Dr Foster. The news that Dr Foster relied on the DH for some of its information generated scepticism about its reliability.
Participants wanted information about specific local services rather than generalized, comparative information. Priorities included information on waiting times, consultants' special interests, their clinical experience and their success rates. This led most to prefer the Dr Foster approach because it was more detailed and locally relevant.
Choice
The extent to which participants thought performance information was useful
depended on their perception of the choice available to them: most were not
confident that they had any. Even if choice were possible, many participants
did not like the idea of 'shopping around' and expected a high
standard everywhere (Box 4).
Participants were divided over whether they would be prepared to travel to a hospital outside their area if they had the opportunity to do so. Some were willing to accept the idea even if it meant going abroad for treatment; others were opposed. Most participants relied heavily on their general practitioner to make decisions on their behalf and they expected the GP to understand and use performance information when making referral decisions.
Accountability
Although initially there was little enthusiasm for performance indicators,
on being shown the material most participants thought it should be publicly
available (Box 5). Some felt that as taxpayers they were entitled to this
information and many supported the notion that healthcare providers should be
required to account for their performance. Others felt this type of
information would be relevant only when they needed it as a patient.
| Box 4 Choice 'What is the benefit of this to us when we are probably stuck with the hospital anyway.' 'If you are going into a maternity ward, you want information about maternity. You don't want anything generalized because it clouds the issue.' 'If you were told you have got the choice between two and one had a 9-week waiting list and one had a week's waiting list, to be honest I'd go for the 9-week waiting list because you know that everybody is going there because he's a good guy.' 'If I needed a serious operation and I knew another hospital could do it quicker than 6 months, I would go.' 'I think it would be a good idea if you had like a portfolio on the doctor, like he's been successful in this and he specializes in that.' 'If the specialist treatment you are going to get is 100 miles away and is better than any treatment available locally, you are going to go for the specialist.' 'If you need to travel to get the best treatment, surely that's better than getting a sub-standard level of treatment.' 'I'd hate it, it's totally inconvenient to have to travel.' 'You rely on your GP to direct you to the consultant he thinks is the best one, most likely a pal of his.' 'I shouldn't have to go shopping around to get healthcare for my son. I would hope that my GP would have that sort of information.' 'You shouldn't have to be surfing the net or looking all over the country for the best place to get treatment, you should be able to get it anywhere.'
|
| Box 5 Accountability 'It's our National Health, we should know it's working and where it's going wrong.' 'We all have to do it nowadays in our jobs. We have to prove what we are doing.' 'But what can you do about it? Tax payments and insurance payments are automatically deducted from your pay; you have no say in how it's spent, so it's a case of getting in touch with your MP.' 'If you are a general Joe off the street you don't look in the papers at these things that are going on in hospitals.' 'I think the people who need it are the people who are going to be able to make changes. To highlight massive problems through the media isn't going to help anybody diddly squat.' 'Doctors can do what they want and I don't even know who they are accountable to. Perhaps the surgery needs to be accountable to someone.' 'Ultimately they [the Government] have got to be the ones responsible because that's where the buck stops.' 'An independent commission because the minute it is in the Government you don't know if that is the real version, if it is the truth.' 'A hospital watch type set-up. I mean we have it on the railways, don't we?' 'The user is the patient. The patients should be able to assess the hospitals.' 'With the advent of computers your GP should have that sort of information at the touch of a button.'
|
Both the DH performance ratings and Dr Foster provide information on mortality rates but very few participants found this information useful. Some questioned its relevance to clinical quality, others found it frightening; but a few felt that publication of such information is essential, citing the Bristol Inquiry into child deaths. There was general agreement that independent assessment is important. Many participants did not trust the Government to arrange this, believing the task should be assigned to an independent commission or consumer 'watchdog'.
| DISCUSSION |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Despite the longer history in the USA of publishing information on the quality of care among different providers, there is as yet little evidence that American patients are using this on a major scale to make choices.3–9 The explanations that have been offered include the following: consumers are not aware of variations in quality so do not seek information about 'the best' providers;5 consumers do not believe they have a choice or prefer to leave it to their employer to choose a plan;10 relevant information is not available at the time it is needed;9 healthcare report cards are badly designed and consumers find them hard to understand;11, 12, 13 consumers do not trust the information or its source.5
Our findings suggest that the British public is likewise ambivalent about the value of performance indicators. Awareness and comprehension of these is currently quite limited, but there is a strong sense that some form of public monitoring is necessary and desirable. Meanwhile the Government is pressing ahead with its plans for wider dissemination of performance information. The justification for this policy may be different in the UK from the USA. While Americans are accustomed to the idea of market competition in healthcare, this notion is more foreign to the British public. British patients have historically tended to rely on their GPs to make choices on their behalf, so it will be interesting to see if the Patient Choice schemes result in greater willingness to choose providers and greater demand for comparative information on the quality of care. Nevertheless, our study suggests that many British people do feel that performance information ought to be available and that the NHS should be seen to be accountable to the taxpayer.
The establishment of the Office of Information on Health Care Performance within the Commission for Health Improvement goes some way towards meeting the demand identified here for an independent source of performance information. It remains to be seen if it will be viewed as more trustworthy than the Department of Health in the eyes of the public.
A key issue is whether greater transparency will enhance or undermine public confidence in the NHS.14,15 To increase the likelihood of beneficial impact, it will be necessary to educate the public and GPs about how to interpret and use the data, and to ensure that choices are available for those who want to exercise them.
| Acknowledgments |
|---|
Note
Picker Institute Europe runs the Advice Centre for the NHS Patient Survey Programme which contributes to the Department of Health's Performance Assessment Framework. Results of the national patient surveys, including those organized by the Picker Institute, are also included in the Dr Foster website.
| REFERENCES |
|---|
|
TOPSUMMARYINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  B. Conner-Spady PhD, C. Sanmartin PhD, G. Johnston MD MBA, J. McGurran MSc, M. Kehler MA, and T. Noseworthy MD MPH Willingness of patients to change surgeons for a shorter waiting time for joint arthroplasty Can. Med. Assoc. J., August 12, 2008; 179(4): 327 - 332. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Singleton Data sources and performance measurement BMJ, October 13, 2007; 335(7623): 730 - 730. [Full Text] [PDF]
|
|
|
|
 |
|
 F. H Zaidi, M. C Corbett, B. J L Burton, and P. A Bloom Raising the benchmark for the 21st century--the 1000 cataract operations audit and survey: outcomes, Consultant-supervised training and sourcing NHS choice Br. J. Ophthalmol., June 1, 2007; 91(6): 731 - 736. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Coulter What do patients and the public want from primary care? BMJ, November 19, 2005; 331(7526): 1199 - 1201. [Full Text] [PDF]
|
|
|
|
|
|
L. M Schwartz, S. Woloshin, and J. D Birkmeyer How do elderly patients decide where to go for major surgery? Telephone interview survey BMJ, October 8, 2005; 331(7520): 821. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. McKee Not everything that counts can be counted; not everything that can be counted counts BMJ, January 17, 2004; 328(7432): 153 - 153. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
