Content uploaded by Merryn Gott
Author content
All content in this area was uploaded by Merryn Gott on Jan 08, 2018
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=yppc20
Download by: [University of Auckland Library] Date: 08 January 2018, At: 11:29
Progress in Palliative Care
Science and the Art of Caring
ISSN: 0969-9260 (Print) 1743-291X (Online) Journal homepage: http://www.tandfonline.com/loi/yppc20
What indicators are measured by tools designed
to address palliative care competence among
‘generalist’ palliative care providers? A critical
literature review
Rosemary Frey, Merryn Gott & Rachel Banfield
To cite this article: Rosemary Frey, Merryn Gott & Rachel Banfield (2011) What indicators are
measured by tools designed to address palliative care competence among ‘generalist’ palliative
care providers? A critical literature review, Progress in Palliative Care, 19:3, 114-124, DOI:
10.1179/1743291X11Y.0000000003
To link to this article: https://doi.org/10.1179/1743291X11Y.0000000003
Published online: 19 Jul 2013.
Submit your article to this journal
Article views: 52
View related articles
Systematic review
What indicators are measured by tools
designed to address palliative care
competence among ‘generalist’palliative
care providers? A critical literature review
Rosemary Frey, Merryn Gott, Rachel Banfield
Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand
Background: There is growing evidence that generalists may lack skills and knowledge in palliative care
provision. This has led to consideration of what the core competencies for palliative care provision among
generalists should be.
Aim: The objective of this review was to present the best available evidence related to indicators of
competence in palliative care provision.
Method: A systematic review of both qualitative and quantitative literature was undertaken. Medline, Medline
in Progress, PubMed and CINAHL databases with additional hand searches of Journal of Palliative Care,
Palliative Medicine, and the International Journal of Palliative Nursing were undertaken for the period
1990–2010. Hawker et al.’s checklist was utilized to select and assess data.
Results: Nineteen of the 1361 articles met the inclusion criteria. The reviewed articles suggest a number of
indicators of palliative care competence including: medical knowledge/skills, perceptions of knowledge/
skills, confidence in palliative care skills, attitudes/opinions towards palliative care, and experience in
palliative care delivery. None of the reviewed research provided definitive evidence as to which indicators
best reflect competency to practice.
Conclusion: Multiple approaches, combined in a strategy of triangulation, must be incorporated in any
appraisal in order to successfully measure palliative care competence.
Keywords: Competence, Generalist, Indicators palliative, Care
Background
Demand for palliative care is predicted to rise exponen-
tially over the next 30 years internationally.
1
Ageing
populations, coupled with an increasing prevalence of
chronic conditions known to be amenable to palliative
care intervention, represent a significant challenge to
policy makers and service planners. Prevalent global
economic trends suggest that significant expansion of
specialist palliative care services is unlikely for most
countries, at least in the short term.
2
This has led to
an increased focus at a policy level upon the role of
‘generalist’palliative care provided by health pro-
fessionals who do not have specialist training in pallia-
tive care and/or work in specialist settings, but who
routinely work with patients at the end of life.
3–5
However, there is growing evidence that generalists
may lack skills and knowledge in palliative care
provision, and numerous studies have concluded by
recommending additional education and training for
this group.
6–12
This has led to consideration of what
the core competencies for palliative care provision
among generalists should be. The End of Life Care
Strategy for England, for example, identifies that
‘One of the most pressing tasks now is to define the
core principles and competence required by each
staff group as they interact with the end of life
pathway’.
5
In the first instance the following compe-
tencies were outlined: communication skills; assess-
ment of needs and preferences; advance care
planning; and symptom control. These were further
developed and a wider context of ‘value and knowl-
edge developments’added.
5,13
Competencies for
specific professional groups have also been developed.
For example, in New Zealand, palliative care compe-
tencies for nurses have been developed along a conti-
nuum from ‘generalist’to ‘specialist’in four
Correspondence to: Rosemary Frey, School of Nursing, Faculty of Medical
and Health Sciences, University of Auckland, ECOM House, 3 Ferncroft
Street Grafton, Auckland. Email: r.frey@auckland.ac.nz
©W.S.Maney&SonLtd.2011
DOI 10.1179/1743291X11Y.0000000003 Progress in Palliative Care 2011 VOL. 19 NO. 3114
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
domains: professional responsibility; management of
nursing care; interpersonal relationships; and inter-
professional health care and quality improvement.
4
A number of training initiatives aimed at improving
clinical performance in relation to these competencies
have now been developed internationally, for example
the NHS Connected Programme, and increased
attention is also being paid to ensuring they are
addressed adequately during undergraduate and post-
graduate medical and nursing programmes for
example in Australia: Palliative Care Curriculum for
Undergraduates.
14
However, such initiatives need to be
underpinned by a strong evidence base and rigorously
evaluated for effectiveness. For this to be achieved,
valid and reliable questionnaire tools are required that
address the core competencies, utilizing appropriate
indicators. The current paper arose from a review under-
taken to critically assess literatures in this field to ident-
ify: (1) what questionnaire tools have been developed to
measure competencies in palliative care provision
among generalists; and (2) the specific nature of the
competencies and indicators measured. The review
hasbeenreportedintwoparts,andinrelationtothis,
has two aims. This paper addresses the second aim;
the first is addressed by Frey et al. (under review).
15
Objective
The objective of this review was to present the best
available evidence related to indicators of competence
in palliative care provision. (A review summarizing the
evidence related to methods of validation of the ques-
tionnaires is described elsewhere.
15
) The specific
review questions to be addressed were:
1. What indicators of competence in palliative care pro-
vision have been reported in the current literature?
2. What domains of competence in palliative care pro-
vision were tapped by these indicators?
3. What methodologies were used to measure compe-
tence in palliative care delivery and what were the
research findings in relation to these indicators?
4. What were the limitations of the included research?
Methods
A systematic review was undertaken to examine the
research literature relevant to indicators of competence
in palliative care provision. The review was conducted
in the following stages: (1) data sources; (2) inclusion
criteria; (3) assessment of relevance; (4) data selection
and appraisal; (5) data synthesis.
Data sources
A list of keywords was developed by consensus among
the reviewers. Relevant research articles (published
between 1990 and 2010) were identified via electronic
searches of Medline, Medline in Progress, PubMed
and CINAHL databases. Relevant articles were also
located through hand searches of the following
journals: Palliative Medicine, International Journal
of Palliative Nursing, and the Journal of Palliative
Care. In addition, references from bibliographies rel-
evant to the search criteria were examined, and cita-
tion indices were followed up.
Inclusion criteria
Inclusion criteria were developed in consultation
between the reviewers. Literature selected for review
had to include an assessment of competence utilizing
a questionnaire. The topic of the research had to be
perceived competence, and the research participants
must have been generalists in palliative care provision.
Further selection criteria limited the scope of the
review to peer-reviewed articles published in English
between 1990 and December 2010. Keywords
included: ‘palliative care’,‘terminal care’,‘end-of-life
care’,‘hospice care’,‘competency’,‘generalist’.
Appropriate wildcards were included to search for
word-ending truncations where needed.
Assessment of relevance and evaluation
of quality
The literature search was performed by one of the
authors (R.B.). Study assessment proceeded through
a systematic sifting process examining the title,
abstract and body of the paper. Independent assess-
ments of paper quality were conducted by the two
reviewers (M.G.) and (R.B.). In the case of a disagree-
ment, consensus was arrived at through discussion. It
was anticipated that an assessment protocol that
could take account of the diverse nature of the research
approaches would be required. Therefore, the
Cochrane study design to weight studies was not
employed. In lieu of this, a method developed by
Hawker et al.
16
was deemed more appropriate for sys-
tematically and objectively reviewing research papers
incorporating both qualitative and quantitative data.
Data selection, appraisal, and synthesis
A checklist adapted from Hawker et al.
16
was utilized
to select and assess data on the: abstract and title;
introduction and aims; methods and data; sampling;
data analysis; bias; results; transferability or generaliz-
ability; implications and usefulness. Statistical analysis
of the study results was deemed inappropriate due to
the diverse nature of the included studies.
Results
A total of 1361 articles were identified and retrieved for
assessment of quality in this review. Of these, 1266 did
not meet the inclusion criteria and were excluded from
the review. Thirty-one articles were retrieved for more
detailed evaluation. Only 19 of the articles were rel-
evant to the identified objectives (competence as a
primary focus, a research design involving a question-
naire, and generalist participants) (see Fig. 1).
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3115
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
Participants
The majority of the studies reviewed (10) assessed per-
ceived competence in doctors, while three measured
competence of nurses. Two articles utilized question-
naires with both doctors and nurses, and four ques-
tionnaires were used to gauge the self-assessed
competence of students.
Review results are presented below as both a
summary and evaluation in relation to each research
objective. Table 1 (Appendix) presents an overview
of the included studies.
Indicators
The studies included in the review utilized a variety of
tools in their assessments of palliative care competence.
Numerous studies included measures of participant
medical knowledge and/or skills to determine compe-
tence in palliative care delivery.
17–21
Self-rated percep-
tions of level of knowledge/competence were also
assessed.
19,22–30
Interestingly, Becker et al.
31
required an
assessment of competency of the professional group
rather than self-assessment in order to reduce social desir-
ability bias. Confidence in addressing various issues in
palliative care was also used as indicator of palliative
care competence.
21,24–26
Related to confidence, many of
the studies included measures of level of comfort/
anxiety.
20,24,25,30–33
Attitudes/opinions related to pallia-
tive care delivery were also assessed.
17,20,22,34
Finally,
measures of experience/frequency in dealing with
palliative care situations were sometimes used.
19,22,27
Domains
Key areas of palliative care (symptom management,
pain, breathlessness, depression, nausea/vomiting
and other common symptoms at the end of life) were
assessed for participant confidence by Dryden and
Addicott.
26
Self-efficacy in palliative care was
measured in two separate studies conducted by
Mason and Ellershaw.
32,33
Domains included: com-
munication, patient management, and multiprofes-
sional team work. Also focussing on communication,
Wilkinson et al.
35
assessed confidence in communi-
cation skills among nurses. Mulder et al.
19
assessed
perceived competence over 18 situations of palliative
care. Areas of measurement included communication
regarding: euthanasia, organ donation, stopping treat-
ment, and the patients’fears. Also covered was treat-
ment of younger patients and those with severe
dyspnoea. Burge et al.,
17
surveyed family medicine
residents for both attitudes and knowledge of pallia-
tive care. The attitudes section of the survey assessed:
communication, preparation for death, relieving suf-
fering, cultural factors, and opioid medication use.
Research by Charlton et al.
25
assessed competence in
the following areas: breaking bad news, empathy, dis-
cussing prognosis, symptom control, and anxiety
about ability in caring. Slatten et al.
29
assessed a
broad range of topics in a 176-item questionnaire.
Items covered were: relief of symptoms and patient-
centred care, competence and dialogue, care of
families and friends, competence in collaboration
across the levels in the organization of services, and
competence in professional development. Using a
smaller questionnaire, Farber et al.
27
surveyed
primary-care physicians for information regarding
the frequency and perceived competence of palliative
care provision. Hughes et al.
18
looked at aspects of
clinical care, communication, and ethical issues of
Figure 1 Flow chart of included literature.
15
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3116
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
Table 1 Summary of included research articles
Author and year Study aims Participants Setting Method Response rate (%) Results
Becker et al. (2007)
31
To explore self-assessed
professional education,
competency, and
educational needs
GPs and registered home
care nurses
Province of Styria,
Austria
Cross-sectional Postal
questionnaire
30 61.8% of 546 respondents felt ‘not at all’
or ‘not sufficiently prepared’for
palliative care by their professional
education (GPs: 70%, nurses: 50.4%).
GPs rated the competency of their
professional guild significantly higher
and their educational needs
significantly lower than nurses
(P<0.01)
Billings et al. (2009)
22
Understand how experience
and attitudes affect
perceived competence
First–fifth year internal
medicine residents
Two academic centres
(University of
Washington and
Medical University
of South Carolina
Randomized control trial of
end-of-life (EOL) care
programme –baseline
assessment online
questionnaire
71 Multivariate model of results
demonstrated a significant
relationship between clinical
experience and self-perceived
competence (P=0.015)
Bradley et al. (2000)
34
To develop instrument to
measure attitudes towards
EOL care
Clinicians from general
medicine, cardiology,
oncology, and geriatric
medicine
Clinicians practicing
in Connecticut
Longitudinal pre-test/
post-test self-report
questionnaire
No information Instrument had acceptable test–retest
reliability and construct validity
Burge et al. (2000)
17
To assess residents’
knowledge and attitudes
towards care of EOL
patients
Year 1 and Year 2 −family
medicine residents
Family medicine
residents at
Dalhousie University
Cross-sectional survey
self-report
questionnaire
94 (Year 1),
86 (Year 2)
Attitude scores were high for both
groups. Concerns regarding
knowledge regarding managing
opioid drugs and symptoms of
dysponea
Buss et al. (2005)
23
Assess perceived
competence of residents to
discuss Advance Directives
(AD) with patients
Internal medicine residents Two university and one
community-based
programme
Cross-sectional self-report
questionnaire
85 Mean perceived competence was 3.8
(range, 1–5). Greater perceived
competence significantly associated
with higher postgraduate year
(P<0.001), having residents
demonstrate exemplary AD
discussions (P<0.001), and less
formal education (P<0.01)
Charlton and Smith
(2000)
24
Ascertain perceived skills
of medical students
Newly qualified doctors United Kingdom Cross-sectional survey
postal questionnaire
24 Mean confidence rating for breaking
bad news =2.9; ability to
empathize =3.2; discussing
prognosis =3.3; providing symptom
control =2.8. Mean anxiety rating in
caring for a dying patient
Charlton et al.
(2000)
25
Ascertain effect of 12 months
spent as GP registrar on
perceived skills in palliative
care
GP registrars West Midlands, UK Longitudinal pre-test/
post-test self-report
questionnaire
61.4 Perceived skills increased over 12
months, lowest skills rated for
scenarios child with leukaemia or
young adult with AIDS
Dryden and Addicott
(2009)
26
Assess impact of study day
on confidence and
knowledge in key areas of
symptom control
Healthcare assistants and
social care officers for
Delivering Choice
Programme
Marie Curie Cancer
Care, UK
Mixed method: pre-test/
post-test follow-up
interviews self-report
questionnaire interview
schedule
80.7 Statistically significant increase in both
knowledge and confidence in
symptom management in palliative
care patients
Continued
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3117
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
Tab le 1 Continued
Author and year Study aims Participants Setting Method Response rate (%) Results
Farber et al. (2004)
27
Involvement and perceived
competence of physicians
in palliative care delivery
Practicing internal
medicine and family
practice physicians
United States AMA
master listing
Cross-sectional survey
postal questionnaire
48 Interest in palliative care was associated
with an increased frequency in
performing palliative care items
(P=0.036). Training in palliative care
was associated with better perceived
performance (P=0.05)
Hughes et al. (2006)
18
Evaluate effects of a palliative
care education programme
on knowledge and
competence
Community nurses Lancaster, UK Longitudinal pre-test/
post-test self-report
questionnaire
99 pre-course,
72 post- course,
51 one year later
Significant increase in confidence and
knowledge
Landmark et al.,
(2004)
28
Evaluate perceived
competence development
in palliative care delivery
Students enrolled in a
2-year postgraduate
programme in palliation
A university college
in Norway
Prospective follow-up design
with five measurements
No information Statistically significant increases in
perceived competence over time
Mason and Ellershaw
(2010)
33
Effect of education
programme on self-efficacy
in palliative care delivery
and fear of death
Two cohorts of fourth year
medical undergraduate
students
University of Liverpool,
UK
Pre-test/post-test self-report
questionnaire
64 Cohort 1
69 Cohort 2
No significant differences in measures
between cohorts in the pre-test.
Within each cohort, statistically
significant post-education
improvements were recorded
in both scales
Mason and Ellershaw
(2008)
32
Effect of education
programme and palliative
care placement on self-
efficacy in palliative care
delivery and fear of death
Fourth year medical
undergraduate students
University of Liverpool,
UK
Mixed method: quantitative
pre-test/post-test self-
report questionnaire
Qualitative Grounded
theory Focus Group
64 Quantitative: significant improvements
in perceived efficacy (SEPC
Communication, t=−16.41,
P<0.001; SEPC −Patient
Management, t=−22.31, P<0.001;
SEPC −Multidisciplinary Teamwork,
t=−15.56, P<0.001). Significant
improvements in thanatophobia were
also recorded (z=−7.51, P<0.001)
Qualitative: three main themes
identified:
•Lack of understanding of the nature
and structure of palliative care
before placement
•Changes to future practice and
relationship with palliative care
specialists
•Difficulty communicating with
patient
Continued
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3118
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
Table 1 Continued
Author and year Study aims Participants Setting Method Response rate (%) Results
Mulder et al. (2009)
19
Effects of a problem-based
palliative care course on
perceived competence and
knowledge
Internal medicine residents Radboud University
Nijmegen Medical
Centre and Canisius
Wilhelmina Hospital,
Netherlands
Longitudinal pre-test/
post-test self-report
questionnaire
86 Limited competence –particularly
in communication. Increased
experience in palliative care situations
positively correlated with perceived
competence. No significant
relationship between perceived
competence and knowledge of
palliative care
Pereira et al., (2008)
20
Evaluate the impact of
palliative care course for
two cohorts on care-related
competencies (knowledge,
attitudes, comfort, skills)
Rural-based family
medicine residents
Universities of Calgary
and Alberta, Alberta,
Canada
Longitudinal pre-test/
post-test self-report
questionnaire observer-
rated OSCEs
80 Significant improvements across four
domains, knowledge, attitudes,
self-perceived comfort scale, and
skills, in two consecutive classes
Shipman et al.
(2008)
21
Evaluate impact of national
palliative care education
and support programme on
knowledge and confidence
District nurses Eight cancer networks
(total 82 primary
care trusts) randomly
selected from
different regions
in England
Mixed method Quantitative:
pre-test/post-test postal
questionnaire qualitative:
framework approach
telephone interviews
focus group
32 Quantitative: small but significant
increase in competence and
knowledge post-course. Qualitative:
data support quantitative results
Slatten et al. (2010)
29
Assess perceived
competence in symptom
relief. Explores factors
affecting good care
routines in palliative care
Clinical nurse specialists
who completed a
postgraduate
programme in palliative
care
Nursing graduates from
a university college
in Bergen and a
university college
in Oslo, Norway
Cross-sectional survey
postal questionnaire
50.6 Results indicated that competencies
dealing with mouth problems, nausea,
anxiety, and the use of the Edmonton
Assessment system had a positive
effect on care routines. Conversely,
the ability to identify lack of care had
a significant negative effect on the
use of care routines
Weissman et al.
(1998)
30
Explore physician trainee
competencies and
concerns in EOL care
Third and fourth year
medical students,
interns and residents
Department of Internal
Medicine, Froedtert
Hospital
Cross-sectional survey self-
report questionnaire
No information Self- reported competence increased
with level of training. All respondents
reported least comfort with
discussions of hydration and feeding
withdrawal. Interns and residents
reported concerns around potential
illegality, breach of ethics and
malpractice in EOL scenarios
Wilkinson et al.
(2008)
35
Evaluate the effectiveness of
a 3-day communication
skills course in changing
communication skills
Registered nurses with at
least 1 year experience
in cancer and palliative
care Patients, who were
eligible if they were able
to understand and speak
English, had no cognitive
impairment and could
give informed consent to
participateinaninterview
with a nurse
10 hospices and a
community nursing
service across
England, Scotland,
Wales and Ireland
Randomized control trial
observer rated +self-
report questionnaires
communication skills
confidence, (State Anxiety
Inventory, General Health
Questionnaire-12, Patient
Satisfaction with
Communication
Questionnaire)
Nurse confidence
scores 85 control
group, 79.7
intervention
group
The communication skills score for the
intervention group increased by 3.4
points post-course but decreased in
the control by 0.05 points
Source:Freyet al. (in press).
15
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3119
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
end-of-life care. Areas of assessment included: case-
load and referrals to palliative care, views of specialist
palliative care services, confidence in practice, meeting
patients’needs, meeting self needs, education received
and education wanted, and knowledge of palliative
care. Landmark et al.
28
measured competencies in
various areas: insight into palliation, ability to use
theory in practice, action competence, professional
role, and cross-professional collaboration. Weissman
et al.
30
also assessed domains of competence in end-
of-life care: communication, management of medical
issues, withdrawal issues, and personal concerns.
Shipman et al.
21
assessed four domains: pain
control, control of other symptoms, onward referral
and emergencies, and general assessment and
support. Each domain had multiple items relating to
that specific topic.
Methodology and findings
The reviewed articles suggest a number of methods for
collecting evidence of competence in palliative care
delivery:
•Course/programme assessment
•Self-evaluation
•Observation/review
•Written quiz/test
Each of these will be described in order.
Course/programme assessment. The most widely uti-
lized method reported in the reviewed articles involved
the evaluation of participation in an education pro-
gramme.
18–21,25,26,28,32,33,35
It is assumed in these
studies that a positive improvement in knowledge,
skills, confidence, and/or anxiety translated into an
improvement in actual competency. Questionnaires
developed by Mason and Ellershaw
32,33
assessed par-
ticipants before and after an educational programme in
palliative care. Five participants were also recruited for
qualitative assessment and one focus group discussion
was conducted. These in-depth qualitative assessments
used a theme-coded approach.
33
Wilkinson et al.
35
administered questionnaires before and after a course
on communication skills in palliative care. Charlton
and Smith
24
surveyed participants on perceived skills
in palliative care delivery on two different occasions
across the 12-month period of general practitioner
vocational training. Hughes et al.
18
assessed knowl-
edge, confidence, and attitudes related to palliative
care before, upon completion, and 1 year after an edu-
cational programme. Landmark et al.
28
measured the
perceived competence of students across five different
periods throughout a 2-year course on palliative care.
A pre-test/post-test method was also used by Mulder
et al.
19
(measuring knowledge and experience) and
Periera et al.
20
(knowledge, attitudes, and self-per-
ceived comfort). Dryden and Addicott
26
also used
pre- and post-questionnaires to measure the effect of
a pilot study day of palliative care provision on knowl-
edge and confidence. In addition, self-selected partici-
pants also took part in a telephone follow-up
interview 6 weeks later.
All studies that measured changes in study measures
for the same group of subjects before and after an edu-
cational course reported improvements. Landmark
et al.
28
measured perceived competence at five points
over a 2-year course in palliative care. Students rated
their competence between some and good competence
at the start of the programme and between very good
and extremely good by the conclusion of the course.
Periera et al.
20
also found upon completion of the
hybrid course on palliative care, improvements in
both perceived competence and observed clinical prac-
tice. Wilkinson et al.
35
found that scores for communi-
cation skills among nurses increased following a
communication skills course in palliative care.
Statistically significant improvements in both self-effi-
cacy and fear of death were recorded when reassess-
ment occurred at the conclusion of a palliative care
course in the study by Mason and Ellershaw.
33
Self-evaluation. Self-evaluation is suggested as one
of the most common forms of competence assessment.
All of the 19 included studies utilized at least one com-
ponent which required participant self-assessment.
Both cross-sectional and pre-test/post-test designs
were employed. The research methodology utilized
by the studies most often required self-assessments of
perceived competence
19,22,23,27,28,30,31
and confi-
dence.
20,21,24–26,35
In some of the included studies par-
ticipants were required to recall the number of times
they had been involved in end-of-life care situations.
Higher scores on these measures are indicators of
greater experience.
19,22,23
Greater experience was
assumed by these studies to translate into greater
competence.
Self-assessments were often conducted in relation to
perceived confidence in palliative care skills.
22,23,25,26
Confidence in these measures generally involved
degree of willingness to perform the listed palliative
care activities.
Results of the studies by both Billings et al.
22
and
Buss et al.
23
found a significant relationship between
clinical experience and self-perceived competence
among internal medical residents. Mulder et al.
19
also found that increased experience through a pallia-
tive care course was positively associated with per-
ceived competence for internal medical resident
participants. However, there was no significant
relationship found between perceived competence
and knowledge of palliative care. In terms of measure-
ment of perceived confidence, Charlton et al.
25
in
studying the effects of 12 months of general practice
vocational training found that confidence in ability
was greater in men than in women, older physicians
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3120
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
in comparison to younger physicians, and registrars
completing training in comparison to newer registrars.
In addition, registrars who had completed formal
training in care of dying tended to report more confi-
dence than those who had not. Similarly, Dryden and
Addicott
26
found a reported increase in confidence
in healthcare assistants and social care officers in
symptom management in palliative care patients
after a training day.
Observation/review. Assessment of competence has
also been attempted using direct observation of pallia-
tive care activities in either a clinical setting
35
or in an
examination setting.
20
In both cases, the assessments
were performed by a neutral party. To this end, check-
lists or other tools were designed. In the study of
Wilkinson et al.,
35
nurse communication skills were
assessed by analysing recordings of nurse–patient
interviews as measured by the Communication Skills
Rating Scale coverage scores. The assessment included
both a score for the nurse’s coverage of nine key areas
of the nursing history as well as a process score. This
process score evaluated the skills nurses used in the
process of the interaction. The instrument was utilized
to assess the effectiveness of a 2-day communication
skills course in changing the nurse participants’com-
munication skills. A patient satisfaction with com-
munication post-interview was also used as a
measure of nurse communication skills. The Patient
Satisfaction with Communication Scale consists of
18 items on an ordinal scale ranging from 0 (strongly
disagree) to 4 (strongly agree).
36
Total scores range
from 0 to 72 with higher scores indicating greater sat-
isfaction. Research by Pereira et al.,
20
utilized three
long Objective Structured Clinical Examination
Stations (OSCEs) with an accompanying score sheet
to assess the effect of a hybrid online/classroom-
based course on palliative care competencies for
family medical residents. The OSCEs (20 minutes
each) were used to assess the impact of the course on
the resident’s skills. The score sheets (comprised of
an itemized checklist and a global fail/borderline/
pass rating scale) covered a number of areas including:
communication, symptom management, and psycho-
logical, social and spiritual needs. The simulated
patients in the OSCEs had both cancer and non-
cancer diagnoses.
The randomized control trial by Wilkinson et al.,
35
found an increase in communication skills scores for
the intervention group after the course, while scores
for the control group decreased. Similarly, Pereira
et al.
20
also found an increase in palliative care skills
post-course for two consecutive cohorts of family
medical residents.
Written quiz/test. A few of the studies included
objective written tests that focused on clinical knowl-
edge within palliative care.
17,18,21
The research by
Burge et al.
17
assessing residents’knowledge about
care of patients at the end of life required respondents
to choose the single best response among multiple
options. This knowledge component contained 25
items. Hughes et al.
18
utilized the Palliative Care
Quiz for Nursing (PCQN) for assessing knowledge
of palliative care.
37
The PCQN consists of 20 state-
ments about palliative care, which require a ‘yes’,
‘no’,or‘don’t know’response. The measure has been
used cross-culturally.
37–40
The questionnaire devel-
oped by Shipman et al.
21
contained three knowledge
assessment questions, the formulation of which was
also based on the work of Ross et al.
37
The study by Burge et al.
17
demonstrated no signifi-
cant difference in knowledge between entering year
one family medicine residents and exiting year two
family medicine residents. Both groups of residents
demonstrated strengths in their knowledge of good
communication skills with patients and families.
However, both groups demonstrated weaknesses in
their knowledge of opioid use and other symptom
management methods. In contrast, results of the
study by Hughes et al.
18
demonstrated an increase in
PCQN scores from a pre-course median of 12.5–15
at the end of the course, which was maintained 1
year later. Finally, nurse participants demonstrated a
small but significant increase in knowledge scores
after participation in an educational intervention,
according to the results by Shipman et al.
21
Research limitations
Many of the studies reported a low response rate, a
clear limitation. Charlton and Smith
24
reported a
response rate of just 24% and Becker et al.
31
30%. In
fact, only 7 of the 19 included studies reported a
response rate of over 70%.
17–20,22,23,26
Becker et al.
31
attributed the low response rate, to a lack of follow-
up reminders as well as questionnaire length (nine
sheets of paper). Small sample size was identified as
another limitation for a number of the studies.
20,28,30
Pereira et al.’s
20
sample included just 16 residents in
2004 and 20 in 2005.
A number of the included studies involved the
evaluation of a continuing education programme.
However, is the measurement of educational outcomes
reflective of competence in palliative care delivery? It
has been argued that participation in continuing edu-
cation does not always ensure competence.
41,42
It
should be noted that with the exception of the study
by Wilkinson et al.
35
measurements of patient out-
comes were not included in study designs.
Limitations of the study conducted by Billings
et al.
22
include a lack of generalizability as solely
internal residents were surveyed. Burge et al.
17
admi-
nistered a questionnaire to two separate cohorts of
residents (Year 1 and Year 2) rather than a single
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3121
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
cohort at the beginning and end of their residency. The
authors’acknowledged that this fact may have limited
their ability to predict whether the observed lack of
change was due to weaknesses in the educational pro-
gramme, rather than a cohort effect. Buss et al.
23
noted that their study was limited by the number of
training programmes assessed. This also led to con-
cerns related to generalizability. Wilkinson et al.
35
also noted that generalizability to other healthcare
professionals was an issue, as their evaluation was con-
ducted solely among nurses.
The longitudinal design employed in the study con-
ducted by Hughes et al.
18
meant there was a possibility
that the sample self-selected for interest in palliative
care. Therefore, participants may have differed in
level of interest from those who did not participate.
Mulder et al.
19
cited a number of limitations including:
lack of comparability of results based on a unique
measure, a low response rate (40%) and an extended
time between tests (1.5 years) giving potential for
other changes to effect results.
Buss et al.
23
drew attention to the discrepancies
between perceived competence and behavioural com-
petence, noting the limitations of self-assessment.
The issue of self-reporting and thus relying on per-
sonal assessment of prior experiences is subject to
many sources of error and bias. This problem is par-
ticularly true in the instance, the use of frequency of
performance of palliative care skills as a measure of
competence. It should be noted that despite their
popularity, questions that require an individual to
report how often they engage in a particular activity
are subject to potential memory errors related to the
passage of time, misattribution, suggestibility, or inat-
tention to name a few.
43
Farber et al.
27
noted the
possibility of cognitive dissonance causing differences
between perceived and actual skills in addition to the
low response rate risking non-respondent bias.
Slatten et al.
29
also noted that self-assessment is an
uncertain form of measurement. The absence of a
control group and the low response rate were also men-
tioned as limitations of their study. The ‘Hawthorne
effect’as a source of error was highlighted by
Charlton et al.
25
who noted that the reported beha-
viours of respondents may be altered by the study
itself.
44
Thus participants may either underestimate
or overestimate their actual skills. Overall, self-assess-
ment is by its very nature subjective and context
dependent. Therefore self-reported abilities will
necessarily vary from actual abilities.
45,46
Measures of perceived competence were often
paired with measures of perceived confidence in per-
forming palliative care services.
22,23,25,26
However;
the two concepts are not interchangeable. According
to a definition provided by Stewart et al.
47
in a study
involving pre-registration house officers:
‘Competent’represented what individuals knew
about their ability and was based on the individ-
ual’s previous experience of the task. ‘Confident’
described a judgment which influenced whether
an individual was willing or not to undertake
an activity. Confidence was not necessarily
based on known levels of competence and there-
fore performance of tasks which were unfamiliar
to the house officer also involved the assessment
of risk. (p. 903)
47
Weissman et al.
30
noted that the next step would be to
correlate perceived levels of competence with actual
performance as a way to evaluate the effectiveness of
an education programme. One method of doing this
is to assess competence in an exam setting such as
the OSCE. A large amount of literature exists on the
use of this method for assessing competency.
48–51
However, one of the primary limitations cited with
this method is that the exam is a simulation and as
such may not reflect the true nature of the work
environment. In other words, the complex real-world
situations that require integrated thinking on the part
of clinicians cannot be tested in such a format.
Discussion
This review used comprehensive search, retrieval, and
review strategies, although searches were limited to the
English language and the grey literature was not
searched. No previous review on this topic could be
identified. What is evident from this review is that a
wide range of opinion exists as to preferred compe-
tency indicators. However, none of the reviewed
research provided definitive evidence as to which indi-
cators best reflect competency to practice. Apparently
then, there is no widely accepted ‘magic bullet’to
determine palliative care competency. Lacking this
definitive evidence, it can be concluded that palliative
care competence can only be implied from the results
of the included research. This is in keeping with the
measurement of any complex construct. According
to Heywood et al.,
52
‘competence’is an intangible
idea. In other words, it cannot be determined by
direct observation, but must instead be inferred from
an examination of indirect evidence such as knowl-
edge, skills, attitudes, and personal attributes. In
such an instance, it would appear that the preferred
approach to assessment should be multidimensional
in nature, incorporating a variety of indicators, and a
strategy of triangulation. By so doing, the collected
inferred evidence of competence would then be more
likely to reflect actual competence. None of the
included research seems to have achieved this goal.
Each of the reviewed approaches comes with its own
strengths and weaknesses. It is our belief that multiple
approaches, combined in a strategy of triangulation,
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3122
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
must be incorporated in any appraisal in order to suc-
cessfully measure palliative care competence.
However, further research is required to determine
what measures should be included and how such a
multidimensional approach could be validated.
Ultimately, the evaluation of palliative care compe-
tence involves a series of trade-offs. A multidimen-
sional approach may serve to balance the weaknesses
found in each approach, enabling us to come closer
to an accurate assessment. However, it must be
remembered, as stated by Kane (p. 164)
53
that ‘the
best we can do is clarify the difficulties inherent in
evaluating professional competence and, perhaps
suggest ways to minimize the impact of these difficul-
ties’. It is hoped that this review has provided a further
step in achieving this goal.
References
1 Gomes J, Cohen L, Higginson I. International trends in circum-
stances of death and dying amongst older people. In: Gott M,
Ingleton C (eds.) Living with ageing and dying: palliative and
end of life care for older people. Oxford: Oxford University
Press; 2011.
2 Gott M, Ingleton C (eds). Living with ageing and dying: pallia-
tive and end of life care for older people. Oxford: Oxford
University Press; 2011.
3 Australian Government Department for Health and Ageing. The
national palliative care strategy –supporting Australians to live
well at the end of life. 2010. Available from: http://www.health.
gov.au/internet/publications/publishing.nsf/Content/ageing-
npcs-2010-toc~ageing-npcs-2010-introduction [accessed 05/01/
2011]
4 Ministry of Health (New Zealand). Gap analysis of specialist
palliative care in New Zealand. Wellington: New Zealand; 2009.
5 Department of Health (England). The end of life care strategy
for England. London: Department of Health; 2008.
6 Hanratty B, Hibbert D, Mair C, Ward C, Capewell S, Livita A,
et al. Doctors’perceptions of palliative care for heart failure:
focus study group. BMJ 2002;325:581–5.
7 Barnes S, Gott M, Payne S, Parker C, Seamark D, Gariballa S,
et al. Communication in heart failure: perspectives from patients
and primary care professionals. J Health Soc Care Comm 2006;
14(6):482–90.
8 Higginson I, Richardson A, Dale J, White P, Murray S, Shipman
C, et al. Scoping exercise on generalist services for adults at the
end of life: Research, knowledge, policy, and future research
needs. Report 2: The literature scoping. Report for the NIHR
SDO Programme. London: NCCSDO, 2007.
9 Gott M, Gardiner C, Payne S, Small N, Seamark D, Halpin D,
et al.. Barriers to advance care planning in COPD. Palliat Med
2009;23(7):642–8.
10 Griffiths G, Ewing G, Rogers M. Moving swiftly on.
Psychological support provided by district nurses to patients
with palliative care needs. Cancer Nurs 2010;33(5):390–7.
11 Gott M, Ingleton C, Bennett MI, Gardiner C. Transitions to pal-
liative care in acute hospitals in England. BMJ 2011;342:d1773.
12 Waterworth S, Gott M, Raphael D, Barnes S. Older people with
heart failure and general practitioners –temporal reference fra-
meworks and implications for practice. Health Social Care
Community 2011;19:1–8.
13 Skills for Care. Skills for care and health national standards.
Available from: http://www.skillsforcare.org.uk/about_us/
contact_us/contact_us.aspx. 2010 [accessed 2011 April 4]
14 Australian Government Department for Health and Ageing,
Palliative Care Curriculum for Undergraduates (PCC4U).
Available from: http://www.caresearch.com.au/Caresearch/
Default.aspx?alias=www.caresearch.com.au/Caresearch/pcc4u.
2010. [accessed 2011 April 4].
15 Frey R, Gott M, Banfield R, Campbell T. What questionnaires
exist to measure perceived competence of generalists in palliative
care provision? A critical literature review. Pall Med; in press
2011.
16 Hawker S, Hardey M, Powell J. Appraising the evidence: review-
ing disparate data systematically. Qual Health Res 2002;12(9):
1284–99.
17 Burge F, McIntyre P, Kaufman D, Cummings I, Frager G,
Pollett P. Family medicine residents’knowledge and attitudes
about end-of-life care. J Palliat Care 2000;16(3):5–12.
18 Hughes PM, Parker C, Payne S, Ingleton MC, Noble B.
Evaluating an education programme in general palliative care
for community nurses. Int J Palliat Nurs 2006;12(3):123–31.
19 Mulder SF, Bleijenberg G, Verhagen SC, Stuyt PMJ, Schijven
MP, Tack CJ. Improved competence after a palliative care
course for internal medicine residents. Palliat Med 2009;23:360–8.
20 Pereira J, Palacios M, Collin T, Wedel R, Galloway L, Murray A,
et al. The impact of a hybrid online and classroom-based course
on palliative care competencies of family medicine residents.
Palliat Med 2008;22:929–37.
21 Shipman C, Burt J, Ream E, Beynon T, Richardson A,
Addington-Hall J. Improving district nurses’confidence and
knowledge in the principles and practice of palliative care. J
Adv Nurs 2008;63(5):494–505.
22 Billings ME, Randall J, Engelberg C, Engelberg RA. Medicine
residents’self-perceived competence in end-of-life care. Acad
Med 2009;84(11):1533–9.
23 Buss MK, Alexander C, Switzer GE, Arnold RM. Assessing
competence of residents to discuss end-of-life issues. J Palliat
Med 2005;8(2):363–71.
24 Charlton R, Smith G. Perceived skills in palliative medicine of
newly qualified doctors in the U.K. J Palliat Care 2000;16(4):
27–32.
25 Charlton R, Field S, Faull C, Campbell I, Harper C, Harper L.
The effect of the general practice registrar year on perceived skills
in palliative care in the West Midlands. Med Educ 2000;34:
928–35.
26 Dryden H, Addicott R. Evaluation of a pilot study day for
healthcare assistants and social care officers. Int J Palliat Nurs
2009;15(1):6–11.
27 Farber NJ, Urban SY, Collier VU, Metzger M, Weiner J, Boyer
EG. Frequency and perceived competence in providing palliative
care to terminally ill patients: a survey of primary care phys-
icians. J Pain Symptom Manage 2004;28(4):364–72.
28 Landmark B, Wahl A, Bohler A. Competence development in
palliative care in Norway: a description and evaluation of a post-
graduate education program in palliative care in Drammen,
Norway. Palliat Support Care 2004;2:157–62.
29 Slatten K, Fagerstrom L, Hatlevik OE. Clinical competence in
palliative nursing in Norway: the importance of good care rou-
tines. Int J Palliat Nurs 2010;16(2):80–6.
30 Weissman D, Ambuel B, Norton AJ, Wang-Cheng R,
Schiedermayer DA. Survey of competencies and concerns in
end-of-life care for physician trainees. J Pain Symptom
Manage 1998;15(2):82–90.
31 Becker G, Momm F, Gigl A, Wagner B, Johann Baumgartner J.
Competency and educational needs in palliative care. Middle
Eur J Med 2007;119(3–4):112–6.
32 Mason SR, Ellershaw JE. Preparing for palliative medicine;
evaluation of an education programme for fourth year medical
undergraduates. Palliat Med 2008;22:687–92.
33 Mason SR, Ellershaw JE. Undergraduate training in palliative
medicine: is more necessarily better? Palliat Med 2010;24(3):
306–9.
34 Bradley E, Cicchetti D, Fried T, Rousseau D, Johnson-Hurzeler
R, Kasl S, et al. Attitudes about care at the end of life among
clinicians: a quick, reliable, and valid assessment instrument. J
Palliat Care 2000;16(1):6–14.
35 Wilkinson S, Perry R, Blanchard K, Linselll L. Effectiveness of a
three-day communication skill course in changing nurses’com-
munication skills with cancer/palliative care patients: a random-
ised controlled trial. Palliat Med 2008;22:365–575.
36 Ware J, Davies-Avery A, Stewart A. The measurement and
meaning of patient satisfaction. Health Med Care Serv Rev
1978;1(January/February):2–15.
37 Ross M, McDonald B, McGuiness J. The Palliative Care Quiz
for Nursing (PCQN): the development of an instrument to
measure nurses’knowledge of palliative care. J Adv Nurs 1996;
23:126–37.
38 Proctor M, Grealish L, Coates M, Sears P. Nurses’knowledge of
palliative care in the Australian Capital Territory. Int J Palliat
Nurs 2000;6(9):421–8.
39 Arber A. Student nurses’knowledge of palliative care: evaluating
an education module. Int J Palliat Nurs 2001;7(12):597–603.
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3123
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
40 Loftus LA, Thompson E. An evaluation of a palliative care
course for generic nurses. Int J Palliat Nurs 2002;8(7):354–60.
41 Blunden G, Langton H, Hek G. Professional education for the
cancer care nurse in England and Wales: a review of the evidence
base. Eur J Cancer Care 2001;10:179–82.
42 Langton H, Blunden G, Hek G. Cancer nursing education: lit-
erature review and documentary analysis. The University of the
West of England. Research commissioned by The English
National Board for Nursing, Midwifery and Health Visiting,
London, 1999.
43 Schwartz BL. Sparkling at the end of the tongue: the etiology of
tip-of-the-tongue phenomenology. Psychon Bull Rev 1999;6(3):
379–93.
44 Franke RH, Kaul JD. The Hawthorne experiments: first statisti-
cal interpretation. Am Socl Rev 1978;43:623–43.
45 Arnold L, Willoughby TL, Calkins EV. Self-evaluation in under-
graduate medical education: a longitudinal perspective. J Med
Educ 1985;60(1):21–8.
46 Wooliscroft JO, TenHaken J, Smith J, Calhoun JG. Medical stu-
dents’clinical self-assessments: comparisons with external
measures of performance and the students’self-assessments of
overall performance and effort. Acad Med 1993;68:285–94.
47 Stewart J, O’Halloran C, Barton JR, Singleton SJ, Harrigan P,
Spencer J. Clarifying the concepts of confidence and competence
to produce appropriate self-evaluation measurement scales. Med
Educ 2008;34(11):903–9.
48 Gessaroli ME, Poliquin M. Competency-based certification
project phase 1: job analysis. Can J Med Radiat Tech 1994;
25(3):104–7.
49 LaDuca A. The structure of competence in health professions.
Eval Health Prof 1980;3(3):253–88.
50 McKane CL, Schumacher L. Professional advancement
model for critical care orientation. J Nurs Staff Dev 1997;
13(2):88–91.
51 McMillan M, Keatinge D, Little P, Madjar I, Bujack E,
Andrews J, et al. Competency standards for the advanced
nurse. Melbourne: Australian Nursing Federation; 1997.
52 Heywood L, Gonezi A, Hager P. A guide to development
of competency standards for professions. Deptartment of
Employment, Education and Training, National Office of
Overseas Skills Recognition. Research Paper No. 7. Canberra
AGP, April, 1992.
53 Kane MT. The assessment of professional competence. Eval
Health Prof 1992;15(2):163–82.
Frey et al. Indicators of competence in palliative care provision
Progress in Palliative Care 2011 VOL. 19 NO. 3124
Downloaded by [University of Auckland Library] at 11:29 08 January 2018
