Content uploaded by Jenny R Billings
Author content
All content in this area was uploaded by Jenny R Billings on Dec 09, 2014
Content may be subject to copyright.
ORIGINAL INVESTIGATION
Jenny Billings
Towards rigour in qualitative health and social research
across European partnerships
Published online: 4 November 2004
ÓSpringer-Verlag 2004
Abstract Undertaking projects with European partners
is an enterprising and rewarding activity, where societies
can start to learn how to transfer knowledge and take
advantage of best practices to further the benefits to
health and social care service users. However, this can
also be a challenging experience, more so when there is
scanty guidance about the best way to conduct Euro-
pean partnership research. Using an on-going study of
integrated care for older persons as an example (the
PROCARE project), this paper provides an account of
the methodological strengths and challenges of the
qualitative approach used. It suggests that factors
assisting a rigorous approach include sound and inclu-
sive coordination, a strong research design, the con-
struction of universally acceptable selection criteria,
detailed interview schedules, and a transversal approach
to data analysis
Keywords European partnership research Æ
Gerontology ÆHealth and social research Æ
PROCARE project ÆQualitative methods
Introduction
This paper provides an account of the methodological
strengths and challenges of qualitative approaches used
in European partnership health and social research. It
focuses particularly on the author’s current involvement
in a study of integrated care for older persons, the
PROCARE project, giving a rationale for the research
design and the plan of investigation, and providing an
account of how challenges to methodological rigour
were pre-empted, identified and tackled.
The use of qualitative methods in European research
Qualitative methods are increasingly being used to ex-
plore health and social issues within a European context.
Commentators believe that there has been a gradual
realisation among policy makers that simply relying on
statistical data sets is insufficient to understand certain
health and social phenomena and develop robust policy
(Fountain et al. 2000). The growth of qualitative re-
search has been particularly evident in the field of drug
use in the European Union, where the European Mon-
itoring Centre for Drugs and Drug Addiction (EM-
CDDA) is increasingly focusing attention on developing
comparable methodologies in qualitative methods across
the EU (Greenwood and Robertson 2000). Other areas
of investigation include HIV/AIDS (Goodwin et al.
2004) and female career moves (Linehan 2001). Not all
of these projects have taken place within a European
collaborative, however; some, for example, have used a
team of investigators from one country and overcome
the language barriers by interviewing in English (Line-
han 2001).
Without doubt there are particular challenges con-
fronting investigators when undertaking research across
European partnerships, where local investigators take
responsibility for the planning and execution of an
agreed research process. These include overall manage-
ment and communication factors particularly when the
projects are large, gaining consensus with plans of
investigation, translation issues, and methodological
comparability through sampling and data analysis pro-
cesses. To heighten this difficulty, there appears to be
little in the way of published methodological details to
guide researchers, resulting in limited tried and tested
examples of process for application to new projects.
The study by Oates et al. (2004) into postnatal
depression does, however, provide some useful details
about management structure, instrumentation develop-
ment and data analysis. This study was concerned
with developing, translating and validating research
J. Billings
Centre for Health Service Studies, University of Kent,
Canterbury, CT2 7NF, UK
E-mail: j.r.billings@kent.ac.uk
Tel.: +44-1227-823876
Eur J Ageing (2004) 1: 73–78
DOI 10.1007/s10433-004-0012-3
instruments that could be used in future studies of
postnatal depression in different cultures and contexts. A
project management team consisting of a member from
each of the participating countries was convened, to
encourage equal dissipation of management account-
ability. All researchers received training in interviewing,
and the management team developed clear data collec-
tion protocols. With data analysis, the transcripts were
subjected to textual analysis in the original language,
concept coding was developed and themes extracted.
Then all the data were collated in one coordinating centre
to ensure that final themes were not only relevant but
accurately reflected the data from each centre.
Oates et al. (2004) acknowledge that, while the
method appeared sound and comprehensive, there were
some limitations to the study, which included the small
numbers of informants in the individual centres, and
what the authors describe as the ‘perhaps controversial
conflation and comparison’ across centres after trans-
lation. The study does, however, provide some useful
insight into methodological processes in European
partnership research.
Following on from this, on-going research into the
treatment of drug dependent offenders in six European
countries (QCT Europe: http://www.kent.ac.uk/eiss/
projects/qct-europe/index.htm) has tackled the issue of
data analysis comparability by employing one researcher
to undertake all the qualitative data analysis. While this
does overcome a potential problem, it could weaken the
impact of country specific data. Constant feedback to
partners to ensure correct interpretation may be a way
forward.
In addition, some lessons from quantitative studies
can provide some transferable management and com-
munication tactics to help inform investigation plans. As
with the Oates et al. (2004) approach, Cook et al. (2002)
agree that partnership projects are best managed by a
small number of experienced professionals at a few
centres. They add that electronic and other communi-
cation vehicles can be used effectively to aid study
management.
Thus while some evidence exists to guide research
plans in qualitative studies, there appear at present too
few accessible and detailed examples to comprehensively
guide project development, particularly in the area of
integrated care for the elderly. This paper provides an
opportunity to build on this deficit through the author’s
involvement in the PROCARE project.
Overview of PROCARE
PROCARE is a collaborative European project funded
by the European Commission’s Fifth Framework Pro-
gramme, involving nine partner countries. It is a 30-
month project due for completion in December 2004.
The countries participating are Austria, Denmark, Fin-
land, France, Germany, Greece, Italy, The Netherlands
and the United Kingdom. With a focus on integrated
care for older persons, the aim of the project is to pro-
vide an overview of health and social care services in the
participating countries, evaluate innovative models and
identify a sustainable model of integrated health and
social care for older persons in need. The project in-
volves a number of phases:
– Phase 1: This has been completed and consisted of
each country providing a National Report that criti-
cally reviewed integrated care services from the
political, organisational and structural viewpoints
(Leichsenring and Alaszewski 2004).
– Phase 2: This empirical part of the study was con-
cerned with identifying and evaluating two case
studies of good practice in the delivery of services and
collaboration between health and social care in each
country. Each model was evaluated from an organi-
sational and user perspective.
– Phase 3: The purpose of this final phase is to under-
take a cross-national analysis. The results will be
presented and discussed with key stakeholders with
the aim of identifying quality indicators for integrated
care services. Recommendations will be made and
results widely disseminated.
The project is currently at phase 3, and partners are
in the throes of data analysis. This paper focuses on
phase 2 and the analysis framework in phase 3 to
highlight how, in practical terms, the research was
planned and conducted to maximise the overall
quality of the project.
It is important at this point to explain the organi-
sational and management structures within this col-
laboration, as this had a bearing on the manner by
which the project worked towards a positive out-
come. The central coordination and management of
the project was organised through the European
Centre for Social Welfare Policy and Research in
Austria. Each of the nine partner countries provided
an academic lead and research team for the country-
specific work packages. There was also a ‘quality
assurance’ team consisting of representatives from
some of the countries to monitor progress. In general,
these organisational processes have promoted a sus-
tainable ‘buy-in’ to the project, encouraged by a
collaborative coordination style. As with most col-
laborations, partners met at intervals in various
European cities to discuss progress. Translation
problems were largely minimised at meetings due to
the verbal fluency of partners and multi-lingual skills
of the lead centre team. Electronic networks were also
established from the onset of the project, encouraging
continuous dialogue.
Plan of investigation and rational for research design
The purpose of the empirical data collection phase of
PROCARE was to reveal the nature and practice of two
integrated care services in each of the nine partner
74
countries from the organisational and user perspective.
Specifically, the objectives of the investigation were: (a)
to describe how services work to provide integrated care,
(b) to explore the experiences of integrated care from the
user and carer perspective, (c) to ascertain the impedi-
ments to effective working and how to overcome them
and (d) to assess the extent to which the service was
person-centred.
The development of the methods was guided by an
initial theory paper generated from the National Re-
ports (Alaszewski et al. 2004) followed by a more de-
tailed methods discussion paper (Billings et al. 2003). An
overview of the design and plan of investigation is given
below, highlighting details of process and indicating
where challenges were envisaged or occurred and efforts
that were made to overcome them.
Research design
Given the European scope of the PROCARE pro-
gramme, it was important to ensure a valid and consis-
tent approach across diverse contexts and environments;
therefore the provision of a suitable research design was
paramount. The most fitting approach was a case study
design which brought together different forms of data
and permitted a robust study of both the objective
characteristics of systems and subjective experiences
across contrasting areas (Yin 2003). Using Yin’s classi-
fication, ‘integrated care’ became the unit of analysis.
There were two case studies per country and 18 case
studies in all (multiple case study design).
Different types of data were bought together to gen-
erate insight into integrated care from a user, carer and
organisational perspective (see Fig. 1 for illustration).
Most of the data sources were qualitative in order to
permit the exploration of experiences, processes and
perceived outcomes within the models (Pope and Mays
2000), and the quantitative data related to the collection
of some simple baseline organisational data and health
profile information of the users taking part. As far as
possible these data sources were intended to be the same
for each case in order to permit meaningful comparison
and ‘replication logic’ (Yin 2003). This serves to
strengthen developing theories across the case studies.
The method of data analysis associated with this
approach uses ‘pattern-matching’ that permits the
blending of different types and levels of data from all the
cases into an explanatory model, purported to enhance
rigour (Yin 2003). The external validity of case study
research is supported through the use of a number of
tactics, in particular the concept of analytical general-
isation, which involves generalising to established theory
(Keen and Packwood 2000; Yin 2003).
It can be seen that this design had some potential for
European collaborative research in having an overall
strong emphasis on blending data permitting a cross
national analysis and consequent inclusive thematic
generation. In this way, it would make allowances for
some natural data imprecision (such as not being able to
accurately match models of integrated care across
countries).
Plan of investigation
An important feature of the planning was to ensure that
common understanding and agreement was reached
regarding key terms. The establishment of criteria be-
came an important feature of the method and consid-
erable discussion at project meetings took place until
consensus was reached. As the meetings were quite large
and consisted of senior academic representatives with
strongly held, diverse views, this was lively at times.
Success was greatly enhanced by small group work of
comparable membership and setting tasks, with timed
feedback to the larger group. During this process the
value of qualitative research and the research design in
particular became apparent in that it permitted the
necessary degree of flexibility and allowed this diversity
of integrated care delivery and context to be embraced
and captured.
Selection of sites
An initial consideration was to achieve a mutual
understanding of what was meant by ‘integrated care’ in
order to ensure some consistency of site selection. What
was clear from the National Reports produced in phase
1 was that models of integrated care were very disparate,
with varying organisational structures, staffing levels
and types, and available resources. Across the European
spectrum concepts applied to these models included
seamless care (Denmark and UK), coordinated care
(France), transmural care (The Netherlands) and joint
working (UK).
However, some key process features common to all
were identified and were included in the selection crite-
ria. One factor was that there had to be an organisa-
Fig. 1 Case study unit of analysis and data sources
75
tional goal of providing integrated health and social care
for older persons; another that there had to be existing
evidence that there was integrated/collaborative working
between health and social care workers/professionals
(i.e. at least one interface between health and social care
provision). Practical criteria were also developed. For
example the models had to be established for longer than
6 months, and the organisation had to be large enough
to support the sampling requirements.
For both the user and carer, and organisational
perspectives, the sampling framework was purposeful in
order to engage those respondents who could inform the
most (Pope and Mays 2000). The number of users and
carers per integrated care model was between 10 and 15
(20–30 per country) and 5 key workers were also inter-
viewed (10 per country). For the organisational per-
spective a focus group of between 6 and 12 staff were
recruited per model (12–24 per country).
There were particular problems to overcome regard-
ing identification of staff role and achieving compara-
bility. With the organisational perspective there were
some fundamental differences between staff types, par-
ticularly in health. Care needed to be taken when using
the work ‘professional’ as it was not easily translated in
other countries, particularly from the health perspective.
Taking nurses as an example, in most countries other
than the UK there is not a professional registration
requirement. Also, there is not an ‘unqualified’ health
care assistant equivalent; in France for example, health
care aides have a diploma equivalent to a nursing
qualification. In the UK there is a clear hierarchy
relating to qualification, but this is not so evident in
other countries. In turn this factor could potentially
have had an impact on the focus group mix and the
ensuing views expressed, and needed to be taken into
consideration.
Data collection
An important but simple factor in ensuring rigour and
gaining agreement at the data collection stage was re-
course to the wider methods literature for guidance about
best approaches. Key data collection issues were agreed
and implemented. This included interviewing carers
separate from users to obtain honest and candid opinions
of integrated care (Burnside et al. 1998) and providing
researchers with key texts on conducting interviews and
focus groups to ensure a common understanding and
application of the process (Morgan 1997; Robson 1993).
The Social Research Update website also proved useful
(http://www.soc.surrey.ac.uk/sru/sru.htm/).
Some variations were encountered with respect to
how the data were to be recorded. Whilst taping of
interviews is considered the approach of choice for full
data capture (Fontana and Frey 2003), this was not
deemed possible by some partners or even permitted in
some countries under certain circumstances. Therefore
some countries taped interviews and focus group ses-
sions, while others took written notes. To combat the
problems with data quality differences the latter
researchers were advised to ensure that notes were read
back to respondents on completion of the interview to
verify completeness and accuracy (Flick 1998). In addi-
tion, the interview schedules were developed to be suf-
ficiently detailed in order to assist in maximising data
capture (see below).
Development of the interview schedules
The interview schedules were developed through identi-
fication of gaps and priorities highlighted within the Na-
tional Reports completed in phase 1 of the research, and
lengthy and intensive discussion. It was felt to be impor-
tant to develop detailed easily understood questions to
avoid difficulties in translating key concepts and themes
between different languages and cultural contexts.
Four schedules were developed for each of the sample
groups (users, carers, key workers and staff focus
groups) that had common question areas but were
applicable to the different configurations and organisa-
tional structures of the models. Broad sections were
developed, within which the detailed questions were
developed. These sections were assessment of need, evi-
dence of person-centred care and joint working, dis-
charge planning and experience of the integrated care
service from the different perspectives.
Questions also covered every eventuality (e.g. related
to home, residential centre or day care facilities and
covering issues around the purchasing of care). This
resulted in a series of optional questions that could be
clearly navigated on the schedule. Despite efforts,
piloting did not take place as it was felt to be too time
consuming and onerous to manage any additional
changes and that the area had been sufficiently debated.
As previously outlined, case study design provided an
analytical framework to enable the development of
explanatory models in relation to integrated care for each
case study. Whilst this may seem appropriate, there were
some challenges in reaching agreement about the actual
data analysis process. Due to the different research
backgrounds of some the partners (such as economics)
ideas ranged from making abbreviated notes from the
data to intense coding and thematic development. Re-
course to the literature confused the issue as there are a
wide spectrum of approaches to choose from ranging
from simple content analysis using pre-determined themes
(Flick 1998) to in-depth familiarising, coding, categoris-
ing and conceptualising (Miles and Huberman 1994).
A factor assisting agreement on the approach was the
method of data collection, which in some instances as
stated, was written notes. This, along with the added
problem of time, disallowed the implementation of more
depth analysis. Ultimately an innovative two stage ap-
proach was developed to firstly organise the data and
secondly to permit the emergence and analysis of themes
across the data—’transversal’ analysis.
76
Stage 1. In order to organise the data into a more
manageable and comparable form a content analysis
approach was adopted along the lines of Flick’s (1998)
guidance. Four analysis protocols were developed using
pre-determined themes that encompassed all the sections
from the four interview schedules. Partners were re-
quired to place data within these themes justified by
actual anonymised quotes, where available. While this
approach signalled the way forward for the European
perspective in this study, partners wishing to do more
detailed analysis were encouraged to conduct this with
their country specific data for local use alongside the
project requirements.
Stage 2. One of the prearranged meetings was used to
identify early emerging ideas from the analysis proto-
cols. Following discussion, different issues coming from
the data were clustered into a number of transversal
themes that appeared in general to have a European,
cross-country applicability: (a) definition and perception
from stakeholder’s view, (b) access and pathways to
integrated care provision, (c) professional cultures and
joint working, (d) key innovations, (e) the role of the
family in integrated care, (f) quality of service; outcome
indicators and orientation towards clients’ needs, (g)
working conditions and (h) context and frame-
work—financing. Partners were then organised into
preferred topic areas, resulting in cross-national teams,
the purpose being to explore and analyse the theme more
widely within the parameters of the project aims through
the collection and analysis of relevant data. These data
were requested from partner countries using a specially
constructed data request sheet.
At the present time colleagues have exchanged
information and are commencing data analysis. The
data exchange has been monitored centrally to minimise
overlap and repetition between themes. The themes have
remained as originally constructed and are in general
relevant. Some aspects of the process have been chal-
lenging and arduous. For example, partners have had to
provide copious amounts of different information on the
request sheets in English by scanning large amounts of
raw data. In order for the teams to analyse the data in a
meaningful way partners are required to produce high
quality translations, which is time consuming but nec-
essary to prevent data sources from losing their impact.
However, despite these difficulties this method is
proving to be innovative and rewarding in terms of
European research. The development of cross national
teams has the potential to reduce interpretive difficulties
and translation errors. Furthermore, it will help to focus
on the chosen issues and to concentrate on comparative
views.
Summary and conclusion
In summary, this project has provided the opportunity
to place the process of qualitative partnership research
in Europe under the microscope and to develop ways of
working towards rigour in ageing research. Factors that
facilitated this included the inclusive coordination and
management processes, which are providing a good
‘steer’ to the project, the construction of universally
acceptable selection criteria, detailed interview sched-
ules, and a transversal approach to data analysis with
cross-national teams. In addition, a strong feature was
the research design which provided a sound underpin-
ning and promoted an understanding of how the partner
projects could be integrated and blended.
Weaknesses of the study included the lack of ade-
quate piloting due to the complexities of managing any
changes to the method, and the inconsistencies between
the richness of data collected and that put forward for
analysis, for which there is not an easy solution. In
addition, although good electronic systems were set up
swiftly, they were in general underused by all but the
central management team, despite encouragement. Al-
though not a weakness, a challenge of the project was
the sometimes necessary but protracted and occasionally
‘angst-ridden’ discussion, especially around developing
definitions and criteria. This was due to the large group
and the importance attached to partner involvement,
and the complexity of the projects stimulating the need
to capture and incorporate every eventuality.
Reflecting on this process, it could be argued that in
order to combat threats to rigour, it is possible to impose
too much structure. Qualitative methods are purpose-
fully designed to encourage the emergence of ideas not
previously planned for, and confining the methods in less
flexible criteria and schedules could reduce this potential.
This could be compounded by the use of pre-determined
themes, which has a tendency to generate data within its
confines, with the result that more meaningful data could
become disassociated from the main body of analysis.
However, alternative methods of handling vast quantities
of data would necessitate adding length and resources to
projects, which, given European commissioning
requirements, may be less acceptable.
Added to this is the fact that condensing large
amounts of country-specific case studies into a European
level analysis will inevitably strip away colourful local
context. However, partners are producing country spe-
cific reports for local application to counteract this. In
general it is perhaps important to strike a balance be-
tween conformity of approach and natural inconsisten-
cies that will result from the variations. Qualitative
approaches are of course ideal to absorb these anomalies
and case study design in particular, in that it promotes
the blending of data and lends itself to a natural com-
parative element.
To conclude, persons joining European academic
partnerships come with a diverse range of expertise,
experience and expectations about what the project is to
achieve, and how it should be done. It is rarely possible to
do the ‘textbook’ approach, but it is possible to work to-
wards this. Achieving success is largely about compromise
to ensure the ‘greater good’, with negotiation and debate
being important elements to overcoming challenges.
77
References
Alaszewski AM, Billings JR, Coxon K (2004) Integrated health and
social care for older persons: theoretical and conceptual issues.
In: Leichsenring K, Alaszewski AM (eds) Providing integrated
health and social care for older persons. A European overview
of issues at stake. Ashgate, Aldershot, pp 53–94
Billings JR, Coxon K, Alaszewski AM (2003) Empirical research
methodology for PROCARE research version 3. Centre for
Health Service Studies, University of Kent, Canterbury
Burnside I, Preski S, Hertz JE (1998) Research instrumentation and
elderly subjects. Image J Nurs Sch 30:185–190
Cook D, Brower R, Cooper J, Brochard L, Vincent J (2002)
Multicentre clinical research in adult critical care. Crit Care
Med 30:1636–1643
Flick U (1998) An introduction to qualitative research. Sage,
London
Fontana A, Frey JH (2003) The interview: from structured ques-
tions to negotiated text. In: Denzin NK, Lincoln YS (eds)
Collecting and interpreting qualitative materials, 2nd edn. Sage,
Thousand Oaks, pp 47–60
Fountain J, Griffiths P, Hartnell R (2000) Qualitative research on
drug use in the European Union. In: Greenwood G, Robertson
K (eds) Understanding and responding to drug use: the role of
qualitative research. EMCDDA scientific monograph series 4.
EMCDDA, Luxembourg, pp 301-308
Goodwin R, Kozlova A, Kwaikowska A et al (2004) Soc Sci Med
(in press)
Greenwood G, Robertson K (eds) (2000) Understanding
and responding to drug use: the role of qualitative research.
EMCDDA scientific monograph series 4. EMCDDA, Luxem-
bourg
Keen J, Packwood T (2000) Using case studies in health services
and policy research. In: Pope C, Mays N (eds) Qualitative re-
search in health care. BMJ Books, London, pp 50–58
Leichsenring K, Alaszewski AM (2004) (eds) Providing integrated
health and social care for older persons. A European overview
of issues at stake. Public policy and social welfare series, vol 28.
European Centre Vienna. Ashgate, Aldershot
Linehan M (2001) Key issues in the senior female international
career move: a qualitative study in a European context. Br J
Manag 12:85–96
Miles MB, Huberman AM (1994) Qualitative data analysis: an
expanded sourcebook. 2nd edn. Sage, Thousand Oaks
Morgan D (1997) Focus groups as qualitative research, 2nd edn.
Sage, Thousand Oaks
Oates MR, Cox JL, Neema S, Asten P et al (2004) Postnatal
depression across countries and cultures: a qualitative study. Br
J Psychiatry 184 [Suppl 46]:s10–s16
Pope C, Mays N (2000) Qualitative methods in health research. In:
Pope C, Mays N (eds) Qualitative research in health care. BMJ
Books, London, pp 1–10
Robson C (1993) Real world research. Blackwell Science, Oxford
Yin R (2003) Applications of case study research, 2nd edn. Sage,
Newbury Park
78
