JRSM > Volume 101, Number 6 > Pp. 313-318

This Article Abstract Figures Only Full Text (PDF) Send a Quick Comment Alert me when this article is cited Alert me when Quick Comments are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Whelan, P. Articles by Bhugra, D. Search for Related Content PubMed PubMed Citation Social Bookmarking                      What's this?

Stress, lies and red tape: the views, success rates and stress levels of the MTAS cohort

Paul Whelan¹ Maja Meerten² Rahul Rao³ Peter Jarrett⁴ Anandamurugan Muthukumaraswamy⁵   Dinesh Bhugra⁶

¹ Specialist Registrar in Old Age Psychiatry, South London and Maudsley NHS Foundation Trust
² ST2 in Psychiatry, Central and North West London NHS Foundation Trust
³ Consultant Old Age Psychiatrist, South London and Maudsley NHS Foundation Trust
⁴ Consultant Psychiatrist and Medical Director, Oxleas NHS Foundation Trust
⁵ Specialist Registrar in Perinatal Psychiatry, South London and Maudsley NHS Foundation Trust
⁶ Professor of Mental Health and Cultural Psychiatry and Dean of the Royal College of Psychiatrists, South London and Maudsley NHS Foundation Trust, Institute of Psychiatry, King's College London, and Royal College of Psychiatrists

Correspondence to: Prof Dinesh Bhugra d.bhugra{at}iop.kcl.ac.uk

	SUMMARY

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 
Objective To track the success rates in obtaining a specialist training (ST) post of a cohort of medical training application system (MTAS) applicants, as well as canvassing their views of the process and measuring symptoms of psychological distress during application and after job allocation.

Design A follow-up cross-sectional survey.

Setting Two large south London psychiatry training schemes.

Participants 101 MTAS applicants.

Main outcome measures Success in obtaining an ST post, respondents' views about MTAS and General Health Questionnaire-12 (GHQ-12) scores.

Results 48 of the original sample responded to the follow-up survey. Data were available about post-MTAS job status for a further 41 trainees. 64% of candidates obtained an ST post. Of those, however, 12% were only offered a fixed-term post. Most unsuccessful candidates were doing locum (34%) or non-training grade (21%) jobs. UK/European Economic Area (EEA) doctors were more successful than non-EEA applicants in obtaining an ST post (odds ratio 5.5, 95% confidence interval 2.1–14.3), as were women candidates compared to men (3.7, 0.5–9.3). The respondents' views about MTAS were globally negative (even if they were successful in obtaining an ST post). The median GHQ-12 score for respondents was 7 out of a potential total score of 12. 79% of the sample scored above the threshold (GHQ-12 4) for psychological distress and 21% experienced significant distress (GHQ-12 8).

Conclusions MTAS was a flawed system. A considerable proportion (36%) of candidates who had been on a Senior House Officer training scheme were left without a training post after MTAS. The system seems to favour UK/EEA applicants. Applicants suffered significant psychological distress during and after the MTAS process.

	Introduction

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 
In the UK, Medical Training Application Service (MTAS) was a single electronic portal for doctors to apply for the specialist ‘run through’ training scheme given the umbrella title of Modernising Medical Careers (MMC). Such single portals have been successfully used in other developed countries for many years because they obviate the need for multiple applications and can support a consistent selection process. The implementation of MTAS in the UK was rushed, undermined by technical failings, and has not been supported by the profession as an effective selection process.¹ The perceived failings of MTAS, particularly with the huge disparity between available training places and the number of applicants, has led to the abandonment of MTAS for the 2008 application round, and to a wholesale review of MMC.

We previously reported the findings of a survey of the views of MTAS applicants from two London psychiatry training schemes, and a request by the Royal College of Psychiatrists for emails from trainers and trainees documenting problems with the system.² The survey was conducted during the application process and before the MMC Review Group decided that the short-listing process was seriously flawed and it was decided to give all applicants (who were in the UK and eligible to work in the UK) an interview with a deanery of their choice (i.e. between MTAS round 1a and 1b).

The MTAS system has met with widespread disapproval from the medical community,^3,4 and the interim report from the MMC inquiry has been released, recommending yet further changes to postgraduate medical education in the UK.¹ Many junior doctors have been affected both practically – a number who had been on the old senior house officer (SHO) rotations have been left without training jobs – and emotionally or in their sense of well-being. A contemporaneous survey had reported that three-quarters of trainees suffered symptoms of psychological distress, and the majority (96%) attributed these to MTAS and/or MMC.⁵

In light of these issues, we decided to re-survey the 101 respondents to the original training scheme survey following the allocation of jobs through round 1 of MTAS.

	Aims

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 

• To track how the cohort of trainees who responded to the original survey had fared following the allocation of jobs.
  
• To re-canvass their views on the process.
  
• To measure symptoms of stress experienced during MTAS application/after job allocation.
  

	Method

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 
The 101 trainees who responded to the original survey were contacted, where possible, using the email addresses from the original survey in July and August 2007, following the announcement of jobs after the first round of MTAS (i.e. rounds 1a and 1b) but before round 2 had started. They were asked if they were successful in obtaining a run-through post and, if so, to provide the details of their appointment. If unsuccessful, they were asked if they had arranged an alternative job and what their plans were for the future. Successful candidates were asked if they obtained a run-through post or a fixed-term specialist training appointment (FTSTA), if their job was in the deanery of their first choice, and whether they had to move to work in another geographical area as a result of the process. Respondents were asked to give their views on the MTAS process as a whole in a free text section (including indicating potential areas of improvement for future selection processes). We wanted to canvass their views after jobs had been allocated because the original survey had been conducted during round 1 of MTAS.

As a result of changed employment a number of email addresses were not available. We used personal contacts to communicate with those trainees without working email addresses, and asked them to complete a paper version of the survey. If it was not possible to make contact with the trainee, we sought information about how they fared in terms of job allocation from the clinical tutors and human resources departments of the Trusts they had worked in.

In addition to the survey's questions, respondents were asked to complete the General Health Questionnaire-12 (GHQ-12) covering the previous four weeks during application and following the announcement of jobs though MTAS. As some of the trainees knew the authors personally, the GHQ-12 was an optional part of the survey. For the same reason, respondents were given the option of completing the GHQ-12 anonymously, and the score was later linked by number (rather than name) to the other survey results.

	Results

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 
Forty-eight trainees responded to the survey. We were able to obtain details on how another 41 candidates fared through the MTAS process from the clinical tutors and human resources departments. Therefore, post-MTAS job status was available for 89 trainees (88.1% of the original sample), as were the views of 48 trainees (47.5%). We were unable to trace 12 (11.9%) of the respondents to the original survey. The demographic details of this sample were broadly similar to those in the original sample, and have been previously described.² The median age was 30 (range 25–49); 57% were men; 29% were born in the UK, 14% in the EEA and 57% outside of the EEA; and the trainees had on average 22 months training experience in psychiatry (range 6–58).

How the applicants fared
Data analyses were conducted using Numbers for Apple Macintosh. Figure 1 is a flow diagram reporting how the cohort of trainees fared in terms of success in obtaining a training post through MTAS, as well as a timeline of the process. Of the 89 candidates for whom we had data regarding their post-MTAS job status, 57 (64.1%) obtained an ST post. Of those, 46 (80.1%) obtained a full run-through and seven (12.3%) FTSTA jobs in psychiatry. One other successful respondent secured an academic psychiatry run-through post, while three changed speciality (two to neurology and one to general practice – all run-through). Forty-three (75.4%) successful candidates got a training post in the deanery of their choice. The 14 (24.6%) who didn't get their first choice had to move considerable distance to work in their non-preferred deanery.

View larger version (10K): [in this window] [in a new window]   Figure 1. MTAS timeline and study flow diagram * Includes 3 who changed specialty and 1 who obtained an academic post

 
Thirty-two (36%) former trainees were unsuccessful in obtaining an ST post. Of those, 11 (34.4%) had arranged or had started locum jobs and seven (21.9%) had taken staff grade posts. Another three unsuccessful MTAS candidates got jobs abroad (one each for Ireland, Greece and Australia). One respondent had decided to take a career break.

Table 1 reports the applicants' success rates in short-listing (original survey) and in obtaining a training post through MTAS (follow-up survey). The candidates who had most success applied for ST2 (64% successful) and ST3 (70% successful). Sixty-eight percent of candidates who were short-listed in round 1a were successful in obtaining a training post compared with 38% who weren't short-listed but were successful in round 1b. Thus the positive predictive value of being short-listed was 67.7% and the negative predictive value of not being short-listed was 62%.

View this table: [in this window] [in a new window]   Table 1. Candidates' MTAS success rates

 
When the results of UK and EEA candidates were pooled into an overall EEA-combined (EEA-c) category, 81.2% of applicants from this group were successful in obtaining an ST post compared to 43.9% of non-EEA candidates. Thus the odds ratio of an EEA-c doctor obtaining a training job compared to a non-EEA applicant was 5.5 (95% CI 2.1–14.3). Interestingly, 80% of women candidates were successful compared to only 50% of men. Thus the odds ratio of a woman obtaining a training post compared to a man was 3.7 (95% CI 1.5–9.3).

Respondents' views about MTAS
The trainees' responses were analysed and categorized by the first two authors. The themes that emerged were ranked by the frequency of response ( Table 2). By far the most frequently cited issue (42% of respondents) was that MTAS was a flawed system. The sample highlighted: technical problems (e.g. system crashes); the lack of importance placed on curriculum vitae (CV) and higher degrees; a lack of transparency of the marking system; a number of competent candidates not getting jobs; and a feeling that jobs were allocated more on the basis of chance rather than the candidate's ability.

View this table: [in this window] [in a new window]   Table 2. Problem areas identified by respondents (n=48)

 
The selection interviews were described as ‘rushed’ and ‘mechanical’. Rather worryingly, two trainees responded that the interview questions had been leaked before round 1b. The themes that emerged from the original survey were repeated in the trainees' responses to this survey. Themes that emerged in this survey that didn't occur in the original survey related to problems with job allocations and concerns about the futures of trainees who obtained FTSTA posts or no job at all. There was greater focus on the broader implications of MMC, the mismatch between the numbers of trainees and consultant posts, and lack of proper consultation with junior doctors about MTAS/MMC. In the original survey many of the candidates, whose views of the MTAS process were predominantly poor, recognized that there were some positives, mainly relating to a need to change the old system. However, in the follow-up survey the mood had changed: the respondents' comments about MTAS were globally negative, even for those who had been successful in obtaining a training job.

Respondents were also asked to recommend improvements for future application processes. Table 3 lists those recommendations ranked by frequency of response. A local deanery-based application was the most frequent recommendation (19% of responses). Respondents wanted an improved application form with more emphasis on CV, degrees, publications and achievements rather than on ‘creative writing skills’. Recommended improvements to the interview process included having lengthier interviews.

View this table: [in this window] [in a new window]   Table 3. Respondents' recommendations for future application processes and training systems (n=48)

 
GHQ-12 scores
Forty-two of the 48 respondents to the survey completed the GHQ-12. Data were analysed using Statistical Package for Social Sciences version 14. The responses were scored using the standard 0-0-1-1 method and a relatively high threshold of 3/4 was used to indicate psychological distress.⁶ Figure 2 is a histogram presenting the candidates' GHQ-12 scores. The median score was seven (range 0–12, IQR 7) and the mode was 12 (the highest possible score on the GHQ-12). Seventy-nine percent of the sample scored above the threshold (4). However, 21% scored eight or above, indicating significant psychological distress possibly warranting treatment. There were no differences between ‘cases’ and ‘non-cases’ in terms of success in obtaining an ST post, nationality, gender (all insignificant ² tests) or age (insignificant Mann-Whitney U test).

View larger version (11K): [in this window] [in a new window]   Figure 2. Candidates' General Health Questionnaire-12 scores (n=42)

 

	Discussion

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 
MTAS was the biggest overhaul in modern times of the way in which junior doctors in the UK applied for training posts. It was introduced in a rushed fashion, without adequate piloting and with insufficient testing of the computer system required to handle tens of thousands of applications, and despite the concerns expressed by many in the profession, including the British Medical Association.⁷ The people most affected (i.e. junior doctors) were not consulted sufficiently during the development of the system. Selection methodology in MTAS was designed for ST1, not for thousands of doctors already working in the specialities.⁸ This is particularly relevant given that the interview component of the selection process at ST1 level has shown a high degree of validity and reliability but may not be transferable to other levels of training.⁹

The respondents to our survey were experiencing considerable levels of psychological distress during and shortly after the MTAS application and job allocation process. Jones et al. found that only 20% of soldiers scored above the same threshold on the GHQ-12 in a large survey of psychological symptoms experienced by the Armed Forces in the UK.¹⁰ The soldiers' median score was one, compared to seven for the doctors in our survey. Avery et al. looked at the mental health of miners following the 1992 national pit closure programme in the UK, finding that the percentage of responders with GHQ-12 scores above the same cut-off point was 46% for those still employed in the mining industry, 52% for unemployed former miners and 22% for working non-miners,¹¹ thus indicating that unemployment contributes to stress.

The proportion of trainees scoring above the threshold was much higher than in a previous study by Clarke et al. looking at stress levels in hospital doctors in New Zealand.¹² They found that psychological distress was associated with life events thereby confirming the role of stressors. The high levels of psychological distress in our sample are in keeping with the recent online survey by Lydall et al., which found that nearly three-quarters of trainees were feeling low energy levels and half were feeling hopeless about their future.⁵ In addition, one third were drinking more and nearly one third said that they were making more mistakes at work.

The sample we report here were doctors who had already commenced specialist training through the previous SHO training scheme system prior to MTAS. Their sense of dissatisfaction with and unfairness of MTAS (as reported in the original survey) was well placed: approximately one-third were left without training posts as a result of the process. Of those who obtained training posts, 12% were left facing the uncertainty of what will happen when their FTSTA contracts expire after a year. Although the two were linked, MTAS was a separate entity to the broader issue of an overhaul of the UK medical education and training system (i.e. MMC). The ongoing discussions and debate about further changes only add to the continuing uncertainty, and are likely to produce further stress: employers and trainers must be aware of these.

Limitations
The numbers involved in the survey are small and represent only two psychiatry training schemes. However, the views expressed are in keeping with those expressed by junior doctors at the MMC inquiry road shows, in the media and through Remedy UK.¹³

The GHQ-12 data need to be interpreted with caution. No baseline scores were available to act as a comparator. It is possible that doctors,⁵ and psychiatry trainees in particular, have high GHQ-12 scores normally given the stressful nature of their work and possible vulnerability factors. Furthermore, GHQ scores for the general population have been reported to be high.^14,15 As only half of the candidates from the original survey responded, a possible respondent bias occurred (i.e. that the candidates who experienced less psychological distress didn't respond to the follow-up survey). We were unable to trace details for 12 (11.9%) respondents to the original survey. Most of those doctors (nine) were not from the EEA. Thus we may have under-reported the plight of non-EEA doctors.

Summary
Despite these limitations, it is evident from this survey that a number of former SHO rotational trainees were disadvantaged by MTAS/MMC, that their views of the process are negative, and that they experienced high levels of psychological distress during and shortly after the application process. However, it is reassuring that a number of the problem areas highlighted by this study were also identified by the MMC inquiry's interim report, and that Professor Tooke's recommendations are similar to those of doctors who responded to the survey.

	Footnotes

 

DECLARATIONS
Competing interests PW gave a talk on MTAS funded by Janssen-Cilag; MM was an MTAS applicant; DB is the Dean of the Royal College of Psychiatrists. PJ, AM and RR have nothing to declare

Funding Department of Health via the National Institute for Health Research (NIHR) Specialist Biomedical Research Centre for Mental Health award to South London and Maudsley NHS Foundation Trust (SLaM) and the Institute of Psychiatry at King's College London

Ethical approval Following discussion with the OXLEAS NHS Foundation Trust Research Ethics Committee it was decided that the study could proceed without the need to undergo formal application through the Research Ethics Committee

Guarantor PW

Contributorship PW was chief investigator, devised the concept, designed the study, collected data, analysed data and wrote the paper. MM contributed to development of the concept, data collection, and paper writing. RR contributed to data collection and analysis, and paper writing. PJ contributed to development of the concept, study design, data collection and analysis, and paper writing. AM contributed to data collection and analysis, and paper writing. DB contributed to concept development, study design, data collection and analysis, and paper writing

	Acknowledgements

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 
None

	References

TOP SUMMARY Introduction Aims Method Results Discussion Acknowledgements References

 

1. Tooke, J, Ashtiany, S, Carter, D et al. Aspiring to excellence. Findings and recommendations of the independent inquiry into modernising medical careers 2007 London: TSO, Available at http://www.mmcinquiry.org.uk/draft.htm (last accessed 10/11/2007).

2. Whelan, P, Jarrett, P, Meerten, M, Forster, K, Bhugra, D. MTAS fiasco: lessons for psychiatry. Psychiatr Bulletin 2007; 31: 425–7.

3. Brown, MJ. Raging against MTAS (UK Medical Training Application Service). BMJ 2007; 334: 549.[Medline]

4. Eaton, L. Government will suspend MTAS for second round of job interviews. BMJ 2007; 334: 1027.[Free Full Text]

5. Lydall, GJ, Malik, A, Bhugra, D. MTAS: Mental health of applicants seems to be deteriorating. BMJ 2007; 334: 1335.[Free Full Text]

6. Goldberg, D, Williams, P. A User's Guide to the General Health Questionnaire 1988 Windsor: NFER-Nelson.

7. British Medical Association. BMA requests urgent meeting with Health Secretary as junior doctors prepare to head overseas 2007 London: BMA, Available at www.bma.org.uk/pressrel.nsf/wlu/SGOY-72JCXL?OpenDocument (last accessed: 31/10/2007).

8. Patterson, F. Fiona Patterson replies to Parashkev Nachez. BMJ 2007; 335: 802.[Free Full Text]

9. Rao, R. The Structured Clinically Relevant Interview for Psychiatrists in Training (SCRIPT): a new standardized assessment tool for recruitment in the UK. Acad Psychiatry 2007; 31: 443–6.[Abstract/Free Full Text]

10. Jones, M, Rona, RJ, Hooper, R, Wesseley, S. The burden of psychological symptoms in UK Armed Forces. Occup Med 2006; 56: 322–8.[Abstract/Free Full Text]

11. Avery, A, Betts, D, Whittington, A, Heron, T, Wilson, S, Reeves, J. The mental and physical health of miners following the 1992 national pit closure programme: a cross sectional survey using General Health Questionnaire GHQ-12 and Short Form SF-36. Public Health 1998; 12: 169–73.

12. Clarke, D, Singh, R. Life events, stress appraisals, and hospital doctors' mental health. N Z Med J 2004; 117: 1204.

13. Remedy, UK. Remedy Submission to the Health Select Committee, London: Remedy UK, Available at http://www.remedyuk.net/index.php?option=com_content&task=view&id=518&Itemid=232&ed=1 (last accessed: 31/10/2007).

14. Department of Health. Health Survey for England 2003 2004 London: TSO.

15. The Scottish Executive. The Scottish Health Survey 2003, Vol. 2 2005 Edinburgh: TSO.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				A. Iversen, B. Rushforth, and K. Forrest How to handle stress and look after your mental health BMJ, April 27, 2009; 338(apr27_1): b1368 - b1368. [Full Text]

This Article Abstract Figures Only Full Text (PDF) Send a Quick Comment Alert me when this article is cited Alert me when Quick Comments are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Whelan, P. Articles by Bhugra, D. Search for Related Content PubMed PubMed Citation Social Bookmarking                      What's this?