Influencing the national training agenda. The UK & Ireland orthopaedic eLogbook

Abstract

A record of operative experience has always been a prerequisite for basic and higher surgical trainees. Although such records are usually examined during trainee assessments and hospital inspections, there has not hitherto been a systematic attempt to interrogate this data, which importantly

Full text

1182 THE JOURNAL OF BONE AND JOINT SURGERY
""
""
ANNOTATION
Influencing the national training agenda
THE UK & IRELAND ORTHOPAEDIC ELOGBOOK
J. L. Sher,
M. R. Reed,
P. Calvert†,
W. A. Wallace,
A. Lamb
From Wansbeck
Hospital,
Northumberland,
England and Faculty
of Health
Informatics, Royal
College of Surgeons,
Edinburgh, Scotland
"
A. Lamb, MBChB, Director
of Web Services
"
W. A. Wallace, FRCS(Ed &
Eng), FRCSEd(Orth),
Professor
Dean of Faculty of Health
Informatics
Royal College of Surgeons of
Edinburgh, Nicholson Street,
Edinburgh EH8 9DW,
Scotland.
"
J. L. Sher, FRCS,
Consultant Orthopaedic
Surgeon
"
M. R. Reed, MD, FRCS(Tr &
Orth), Consultant
Orthopaedic Surgeon
Department of Orthopaedics
and Trauma Surgery
Wansbeck Hospital,
Woodhorn Lane, Ashington,
Northumberland NE63 9JJ,
UK.
"
† P. Calvert, deceased, Past
SAC Chairman and
Consultant Orthopaedic
Surgeon
St George’s Hospital,
London, UK.
Correspondence should be
sent to Mr L. Sher; e-mail:
lester.sher@btinternet.com
©2005 British Editorial
Society of Bone and
Joint Surgery
doi:10.1302/0301-620X.87B9.
16433 $2.00
J Bone Joint Surg [Br]
2005;87-B:1182-6.
A record of operative experience has always
been a prerequisite for basic and higher surgi-
cal trainees. Although such records are usually
examined during trainee assessments and hos-
pital inspections, there has not hitherto been a
systematic attempt to interrogate this data,
which importantly reflects the day to day,
“coalface” experience of trainees.
There is no published data on the operative
experience of a UK or Irish orthopaedic trainee
before specialist registration. There is a wide-
spread belief that the number of operations
performed during specialist training is decreas-
ing. Factors such as reduction in junior doctors
hours of work and diversion of work away
from training centres have major effects on
patterns of work.
The development of an elogbook has pro-
vided hard data on trainees’ experience and
permitted insights into the “in training oper-
ative experience” of trainees in trauma and
orthopaedic practice in the United Kingdom
and Ireland and allows detailed analysis of
the performance of trainee, trainer, hospital
and training programme at the click of a but-
ton.
Background
The project began in the Northern deanery in
1998 and was adopted as a collaborative ven-
ture in 2000 by the British Orthopaedic Associ-
ation (BOA) Education Committee, the Spe-
cialist Advisory Committee (SAC) in Trauma
and Orthopaedics, the British Orthopaedic
Trainees Association (BOTA) and the Royal
College of Surgeons of Edinburgh (RCSEd).
Funds were raised from the BOA, the Editorial
Board of the
Journal of Bone and Joint Surgery
,
the Charnley Trust, the Wishbone Trust, Smith
& Nephew, Johnson & Johnson and Biomet.
Over several years a committed group of
trainees and trainers tested several versions of
the logbook leading to the current product.
Responsibility for the project has passed to the
BOA eLogbook Validation & Authorisation
Committee (eVAC) and the current software
was produced and is maintained by the Faculty
of Health Informatics at the RCSEd.
Details of the elogbook
Current options for elogbook entry
(Fig. 1)
.
Surgeons’ eLogbooks
1
can be designed as ‘thin’
or ‘thick’ client applications. ‘Thin’ clients rely
Hospital terminal
minimal computer facilities
Main server
at RCSEd
Personal computer
full computer facilities
Fig. 1
A diagram showing how data synchronisation is achieved. The computers ‘talk’ to each
other to check that their data is identical. If not, data is transferred by the main server at the
Royal College of Surgeons of Edinburgh (RCSEd).

INFLUENCING THE NATIONAL TRAINING AGENDA 1183
VOL. 87-B, No. 9, SEPTEMBER 2005
on a browser to surf the Internet (e.g. Internet Explorer or
Netscape Navigator) and its most important advantage is
that no software need be downloaded onto the user’s com-
puter. This module avoids software conflicts but relies on a
live connection to the Internet with the information held on
a remote computer, in this instance at the Royal College of
Surgeons, Edinburgh.
A ‘thick client’ module is software loaded onto the user’s
computer by CD or Internet. It is faster because it uses only
one computer and does not require a constant Internet con-
0
10
20
30
THR
Year 1
A S-TS S-TU P T
Operative supervision
0
10
20
30
THR
Year 2
A S-TS S-TU P T
Operative supervision
0
10
20
30
THR
Year 3
A S-TS S-TU P T
Operative supervision
0
10
20
30
THR
Year 4
A S-TS S-TU P T
Operative supervision
0
10
20
30
THR
Year 5
A S-TS S-TU P T
Operative supervision
0
10
20
30
THR
Year 6
A S-TS S-TU P T
Operative supervision
0
50
100
THR
Years 1 to 6
A S-TS S-TU P T
Operative supervision
National Trainee X
Fig. 2
How training in total hip replacements (THRs)
was supervised, by seniority of trainee from
years 1 to 6 - Trainee X is compared with th
e
national mean for trainees in the same year in
training (A, assisted only; S-TS, surgeon super
-
vised by trainer scrubbed; S-TU, surgeon
supervised by trainer unscrubbed; P, per
-
formed with no in-theatre supervision; T
,
trained by a colleague).

1184 J. L. SHER, M. R. REED, P. CALVERT, W. A. WALLACE, A. LAMB
THE JOURNAL OF BONE AND JOINT SURGERY
nection. The RCSEd ‘thick client’ eLogbook is uniquely
designed to automatically synchronise with the ‘thin client’
module thus keeping both versions up to date.
As more people get broadband connections at hospital
and home, the ‘thin’ clients will be more advantageous but,
at present, their use in isolation could exclude many users.
The suite of modules has eLogbook programmes for the
Palm and Pocket PC versions of personal digital assistants
(PDA). All the modules are compatible and any or all ver-
sions can be chosen.
Procedure classification.
After much debate, a system was
devised to encompass the information needed by the United
Kingdom and Irish SAC. Users can submit suggestions for
unlisted procedures, which once ratified by the eVAC com-
mittee, appear seamlessly as the users’ ‘Synchronisation’
button is next pressed. The great majority of users’ sugges-
tions have been incorporated already.
Data protection.
Because data which is defined as ‘sensitive’
or ‘confidential’ by the UK Data Protection Act
2
is collected
in the logbook, each user must register with the data pro-
tection authorities as a ‘data controller’. Although all log-
books are password-protected, users are responsible for the
safekeeping of their data. The RCSEd server uses the same
level of encryption security as bank web sites and the data
is stored simultaneously on two servers which are regularly
backed up off-site. Each user owns their data and collated
information is administered by the eVAC committee.
Information provided.
The eLogbook gives information on
levels of supervision (Fig. 2) and training opportunities pro-
vided by specific trainers, hospitals and training pro-
grammes. This is stratified according to a trainee’s seniority
(Fig. 3).
In order to compare training posts more accurately, a
number of reference operations or operation groups have
been established and trainees can thereby be compared with
their peers. The example in Figures 2 and 3 shows total hip
replacement (THR) and the numbers of training operations
under each supervision code, thereby defining the use of
available opportunities.
Such analysis allows comparison of a trainee’s experience
in a given time period or collection of procedures (e.g.
hand, foot and ankle, spine) with the national average.
Training opportunities offered by training programmes,
hospitals or trainers can also be compared with national
figures. Such comparisons display not only total numbers of
procedures but also identify unused potential learning
experiences.
Access to the reports is restricted to defined users. Train-
ees have access to their own and pooled national compara-
tive data. Training programme directors can examine a
local individual’s performance and individual trainers and
hospitals. The SAC chairman has access to all regions and
all training departments.
Progress
Since 1 October 2003, it has been compulsory for all spe-
cialist registrars in the United Kingdom and Ireland to sub-
mit data electronically to the Trauma & Orthopaedic
Logbook.
1
Although uptake rapidly increased during 2003,
compliance is now 92%. Trainees are also encouraged to
upload voluntarily their historical operative data.
Data collection and analysis.
Trainees were categorised by
year-in-training (YIT) based on the time since appointment
to a training scheme. If the trainee reported their date of
completion of specialist training, this date was used prefer-
entially to calculate their YIT group.
Only operations performed during 2004 and uploaded
prior to April 2005 were included for analysis. This paper
represents a snapshot of training activity in 2004. A list of
eligible specialist registrars (SpR) and YIT for each opera-
tion is created.
Each trainee’s work is analysed by YIT. Any trainee year
with fewer than 150 operations is excluded on the basis of
poor compliance. The process forms a ‘pot’ of operations
100
90
80
70
60
50
40
30
20
10
0
Operations (%)
Hip
replacement
Knee
replacement
External
fixation
All hands All
paediatrics
All trauma All
elective
Year 1
Year 2
Year 3
Year 4
Year 5
Year 6
Fig. 3
Does trainer practice evolve with maturing trainees? The percentage of cases where the trainee performed the surgery, for each
year in training.

INFLUENCING THE NATIONAL TRAINING AGENDA 1185
VOL. 87-B, No. 9, SEPTEMBER 2005
for analysis divided by category such as region, hospital, or
trainer.
With regard to the level of supervision, a trainee must be
present at the documented operations. Five levels of
involvement are available (Table I). The level of involve-
ment is categorised as assisted or performed, with or with-
out the trainer’s presence. A trainee must have carried out
at least 70% of the operation procedure in order to enter
‘performed’. This ensures that a trainee performing a THR
(implanting both the components) or TKR (implanting
both the femoral and tibial components) must carry out the
majority of the operation.
Regions.
Five regions were defined in order to assess the
location of a training scheme with regards to operative
experience (Table II).
Results
By April 2005, 1509 users were registered on the website of
whom 999 stated they were specialist registrars. A total of
906 of these were confirmed by their programme director as
specialist registrars and this cohort forms the basis of the
Table III. Operative experience of the average UK trainee in 2004; mean number of operations performed by the trainee in
each training year
Number of operations Missed
opportunity* (%)Operation group Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Total years 1 to 6
Hip replacement 4.8 6.4 5.9 5.2 7 7.9 37.2 55
Knee replacement 4.6 7.5 6.2 5.6 7 12.5 43.5 48
External fixation 0.7 1.3 1.1 1.4 1.3 1.1 6.8 30
All hands 18.1 19.7 15.6 22.2 16.7 11.1 103.6 26
All paediatrics 4.6 4.4 5.8 8.3 4.2 6.7 34 32
All trauma 76.4 86.3 71.4 80.4 65 55.6 435.1 16
All elective 54.8 65.9 59.5 65.8 68.2 90.8 405 43
* where the trainee was present but did not perform a significant part of the operation
Table IV. Levels of performance of operations based on region classification. Each column is the mean
number of operations attended by all trainees analysed for that region during six years of 2004 training
London North England Scotland South England Other National
Hip replacement
Performed 25 34 49 40 56 37
Assisted 48 44 50 41 57 46
Knee replacement
Performed 39 42 44 45 54 43
Assisted 48 39 31 40 34 40
External fixation
Performed 4 7.7 5 6.2 9.6 6.8
Assisted 1.5 3.7 2 2 3.1 2.9
All hands
Performed 101 110 87 100 91 104
Assisted 42 39 32 30 32 36
All paediatrics
Performed 24 36 22 36 38 34
Assisted 16 15 10 18 13 16
All trauma
Performed 372 436 339 462 504 435
Assisted 69 86 64 86 79 80.5
All elective
Performed 418 430 334 395 406 405
Assisted 342 298 255 318 248 303
Table I. Categories of operative supervision in the orthopaedic eLogbook.
For this analysis ‘performed’ = S-TS+S-TU+P (*the trainee must have
performed more than 70% of the operation’)
Category Level of supervision
A Assisted only
S-TS Surgeon supervised by trainer scrubbed*
S-TU Surgeon supervised by trainer unscrubbed
P Performed with no in-theatre supervision
TTrained a colleague (usually a more junior surgeon)
Table II. The five regional groups devised to assess location on training
programmes on operative experience
Region Training programme
London/Thames North East Thames (Middlesex/University College
Hospital, Percival Potts, Royal London, Stanmore),
South East Thames, North West Thames
Northern England Leicester/Trent, Mersey, North Western, Northern,
Nottingham/Trent, Sheffield/Trent, West Midlands
(Birmingham), West Midlands (Oswestry), Yorkshire
Scotland East of Scotland, North of Scotland, South East
Scotland, West of Scotland
Southern England Eastern, Oxford, South Western Region (Bristol),
Devon and Cornwall, Wessex
Others Wales, Northern Ireland, Ireland and Armed Forces

1186 J. L. SHER, M. R. REED, P. CALVERT, W. A. WALLACE, A. LAMB
THE JOURNAL OF BONE AND JOINT SURGERY
analysis. Within trauma and orthopaedic surgery in the
United Kingdom and Ireland there are approximately 1700
consultants and 760 staff and associate specialist (SAS) doc-
tors, who can also use the software if they wish. Each user is
recognised within the system through a unique General
Medical Council number (both UK and Irish). Although the
database now includes over 500 000 operations, the 2004
data represents 157 492 uploaded operations.
Table III shows a snap-shot of the current operative expe-
rience which United Kingdom and Irish trainees currently
have. This shows the experience in each training year for
the average trainee and represents operations performed by
the trainee. Missed opportunities where the trainee was
present but did not perform the surgery are also shown.
Figures 2 and 3 represent the National Data and show
the evolving level of supervision as trainees pass from years
1 to 6. Figure 2 demonstrates how a trainee compares with
the mean national data with regard to operative experience
of THR. A similar histogram could be produced comparing
an individual trainee with peers in a hospital or within a
deanery. Figure 3 shows the percentage of operations where
the trainee performs the surgery.
Table IV shows the number of procedures which a
trainee can expect to perform and assist with during six
years of training across the different regions.
Discussion
Although there is only a small difference in the number of
trauma
versus
elective procedures during six years of
Higher Surgical Training there is a striking difference in the
percentage of procedures performed, 84%
vs
57%. This
could imply that trauma operations are easier, or that there
is a different attitude to the hands-on supervision of trauma
surgery.
Even with common elective operations, such as THR and
TKR, up to half the potential training opportunities are
missed and the trainee appears to be an assistant rather
than a surgeon-in-training. It is questionable whether good
training involves trainees repeatedly assisting their trainer
doing the same operation, such as THR or TKR, 45 times,
the current mean for all trainees in this study.
It is alarming that each trainee on average is performing
less than ten THRs per year during their training. While the
authors accept that a procedure classified as assisting could
involve the trainee performing up to 70% of the operation,
this is hardly enough practical exposure to the most com-
mon elective operations to reassure the public that the
trainee will be competent at the end of their training.
The results in Figure 3 suggest a subtle evolving pattern
of increasing trainee performance with regard to THR and
TKR but not in hand or paediatric practice. Trauma sur-
gery from the outset provides intensive experience which
changes very little with seniority.
An impression of the effect of shortening higher surgical
training which has been suggested by the Department of
Health is gained from Table III. If training were to be short-
ened to four years at current training intensity, it would be
to reduce the number of THRs and TKRs which were per-
formed from 37 and 44 to 22 and 24, respectively, for the
whole training programme. It would also reduce elective
exposure by 39% and trauma by 28%.
There is no clear view of the minimum number of pro-
cedures a surgeon should perform before accreditation, nor
how many operations a trainee should assist before they
perform one. This is likely to vary according to the com-
plexity of the procedure and the aptitude of the trainee.
However, this is unlikely to explain the whole picture as
trainees about to become consultants still perform less than
half the THRs they attend. Our data suggest that, were
thresholds to be set, a significantly greater number of
trainee-performed operations could be achieved within cur-
rent arrangements.
With shorter training it may be the time to stipulate the
number of procedures a trainee should perform before
completion of training. We have polled European ortho-
paedic associations and, of respondents, found Germany,
Hungary, Norway, Spain and Switzerland already do this.
Croatia, Ireland, the Netherlands, Sweden and the United
Kingdom do not.
Examination of individual deanery performances indi-
cates more variation than pooled regional data. With time,
an ‘operative profile’ of each training post will be obtained,
allowing programme directors to build equity into their
programmes and address trainer weakness proactively. This
more confidential aspect of the logbook is restricted to the
SAC and individual programme directors.
The eLogbook can be a highly sensitive barometer of the
training experience of UK and Irish trainees. This was high-
lighted by the recent identification of a 20% fall in trainee
exposure to THR,
3
believed to be related to the European
Working Time Directive.
4
With the United Kingdom Government set on re-organ-
ising postgraduate medical education through the Post-
graduate Medical Education Training Board and
shortening training times, it is imperative that missed train-
ing opportunities are minimised. Trainees towards the end
of training should be ideally performing rather than assist-
ing at most routine surgery and, where possible, offering
supervised training to those at the very beginning of their
surgical career. We believe this is not the case at present in
the United Kingdom and that the present Department of
Health initiatives are having a disturbing effort on training.
References
1. Website for the Electronic Logbook.
http://www.rcsed.ac.uk/logbooks (accessed
06/06/05).
2. Data Protection Act 1998.
http://www.opsi.gov.uk/acts/1998/19980029.htm
(accessed 20/06/05).
3. Newman M.
Fears mount for juniors’ training.
Hospital Doctor
27 January 2005:1.
4. European working time directive.
http://www.dh.gov.uk/policy and guidance/
human resources and training/working differently/European working time directive
(accessed 20/06/05).