Content uploaded by Rifat Atun
Author content
All content in this area was uploaded by Rifat Atun
Content may be subject to copyright.
Policy Forum
Guidance for Evidence-Informed Policies about Health
Systems: Rationale for and Challenges of Guidance
Development
Xavier Bosch-Capblanch
1,2
*, John N. Lavis
3
, Simon Lewin
4
, Rifat Atun
5
, John-Arne Røttingen
4,6
, Daniel
Dro
¨schel
1,2
, Lise Beck
1,2
, Edgardo Abalos
7
, Fadi El-Jardali
8
, Lucy Gilson
9
, Sandy Oliver
10
, Kaspar Wyss
1,2
,
Peter Tugwell
11
, Regina Kulier
12
, Tikki Pang
12
, Andy Haines
13
1Swiss Tropical and Public Health Institute, Basel, Switzerland, 2University of Basel, Basel, Switzerland, 3McMaster Health Forum, Centre for Health Economics and Policy
Analysis, Department of Clinical Epidemiology and Biostatistics, and Department of Political Science, McMaster University, Hamilton, Ontario, Canada, 4Norwegian
Knowledge Centre for the Health Services, Oslo, Norway, and Health Systems Research Unit, Medical Research Council of South Africa, Cape Town, South Africa, 5Global
Fund to Fight AIDS, Tuberculosis and Malaria, Geneva, Switzerland, 6Harvard Kennedy School, Cambridge, Massachusetts, United States of America, 7Centro Rosarino de
Estudios Perinatales, Rosario, Argentina, 8Department of Health Policy and Management, American University of Beirut, Beirut, Lebanon, and McMaster Health Forum,
McMaster University, Hamilton, Ontario, Canada, 9School of Public Health, University of Cape Town and Department of Global Health and Development, London School
of Hygiene and Tropical Medicine, United Kingdom, 10 EPPI-Centre, Social Science Research Unit, Institute of Education, London, United Kingdom, 11 Centre for Global
Health, Institute of Population Health, University of Ottawa, Ottawa, Ontario, Canada, 12 Innovation, Information, Evidence and Research, World Health Organization,
Geneva, Switzerland, 13 Departments of Social and Environmental Health Research and of Nutrition and Public Health Research, London School of Hygiene & Tropical
Medicine, London, United Kingdom
This is one paper in a three-part series that
sets out how evidence should be translated
into guidance to inform policies on health
systems and improve the delivery of clinical
and public health interventions.
Introduction
Present trends suggest that many of the
poorest countries in the world, including
many in sub-Saharan Africa, will not meet
the health-related Millennium Develop-
ment Goals [1] (MDGs), especially MDG
4 (reducing under-five mortality) and
MDG 5 (reducing maternal mortality)
[2]. Even in those countries that are on
track to meet health MDGs, striking
inequities exist among countries and
among socioeconomic groups within them
[3], despite effective and cost-effective
interventions being available to improve
population health, including that of vul-
nerable groups [4]. Such interventions are
delivered through health systems, which
consist of ‘‘all organisations, people and
actions whose primary intent is to pro-
mote, restore or maintain health’’ [5], but,
in many settings, interactions between
weakened health systems and the some-
times conflicting demands of single-disease
intervention programmes are hindering
the uptake and implementation of life-
saving interventions [6–8]. A growing
number of governments, international
institutions, and funding agencies have
therefore recognised the urgent need to
coordinate and harmonise investments in
health systems strengthening in low- and
middle-income countries (LMICs) to pro-
vide universal social protection and effec-
tive coverage of essential health interven-
tions [9].
Investments in health systems aim to
‘‘enhance [their] performance…for meet-
ing the needs of patients and populations
in an equitable and efficient manner’’ [10]
while reducing the risk of impoverishment
due to the costs of care [11]. However,
although a number of broad principles
have been proposed [12], there is no wide
agreement on the operational definition of
health systems strengthening [13], and it
remains unclear how health systems can
best be strengthened. Because the evi-
dence base addressing this issue is patchy
[14], health systems research has recently
been identified as a priority [15] and its
definition and scope have been outlined
[16]. Indeed, the need for greater capacity
to produce evidence to inform health
systems strengthening was one of the
drivers that led to the first global sympo-
sium on health systems research (Mon-
treux, Switzerland, November 2010) [17],
at which some of the issues developed in
this article were presented and discussed.
Importantly, to be useful to policy
makers, research evidence needs to be
retrieved, its quality appraised, and the
recommended options properly framed in
The Policy Forum allows health policy makers
around the world to discuss challenges and
opportunities for improving health care in their
societies.
Citation: Bosch-Capblanch X, Lavis JN, Lewin S, Atun R, Røttingen J-A, et al. (2012) Guidance for Evidence-
Informed Policies about Health Systems: Rationale for and Challenges of Guidance Development. PLoS
Med 9(3): e1001185. doi:10.1371/journal.pmed.1001185
Published March 6, 2012
Copyright: ß2012 Bosch-Capblanch et al. This is an open-access article distributed under the terms of the
Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any
medium, provided the original author and source are credited.
Funding: The development of the Handbook was supported by a grant to the WHO by the Rockefeller
Foundation, which had no role in study design, data collection and analysis, decision to publish, or preparation
of the manuscript. The paper represents the views of the authors and neither WHO nor the Rockefeller
Foundation.
Competing Interests: The Swiss Tropical and Public Health Institute and the Norwegian Knowledge Centre
for the Health Services received funds from the WHO for the contributions of DD, LB, SL, and XBC to developing
a Handbook to produce health systems guidance, and some of this work is reported in this article. All other
authors declare no competing interests.
Abbreviations: AGREE, Appraisal of Guidelines Research and Evaluation; LMIC, low- and middle-income
country; MDG, Millennium Development Goal; WHO, World Health Organization
* E-mail: x.bosch@unibas.ch
Provenance: Not commissioned; externally peer reviewed.
PLoS Medicine | www.plosmedicine.org 1 March 2012 | Volume 9 | Issue 3 | e1001185
the form of guidance. In an analogy with
clinical practice guidelines (‘‘systematically
developed statements to assist practitioner
and patient decisions about appropriate
health care for specific clinical circum-
stances’’ [18]), we define health systems
guidance as systematically developed state-
ments produced at global or national levels
to assist decisions about appropriate op-
tions for addressing a health systems
challenge in a range of settings and to
assist with the implementation of these
options and their monitoring and evalua-
tion (Box 1). We use the term ‘‘guidance’’
rather than ‘‘guidelines’’, as health systems
and the evidence on health systems are
highly context sensitive. Health systems
guidance statements refer to policy options
that are accompanied by assessments of
the quality of evidence supporting them
and the potential for unintended harms,
and by discussions of implementation and
contextual issues.
The need for evidence-informed guid-
ance on policies that impact health systems
performance is widely accepted [19] and is
one of the six priorities of the director-
general of the World Health Organization
(WHO) [5]. Moreover, a World Health
Assembly resolution [20] recently urged
member states to use evidence-based
approaches to assess ‘‘country’s health
and health systems challenges’’ and to
develop ‘‘evidence-based responses to
evolving challenges and opportunities,
and to involve all relevant stakeholders’’.
However, although well-established meth-
ods exist to develop clinical guidelines
[21], there is little experience in develop-
ing health systems guidance and the
process poses conceptual and methodolog-
ical challenges related to the different
types of evidence to be considered, the
complexity of health systems and the pre-
eminence of contextual issues. The current
experience is mainly related to the devel-
opment of different by-products of re-
search syntheses and decision aids [22]
targeted towards policy makers, rather
than systematically and transparently de-
veloped guidance.
This paper, which is the first in a three-
part series on health systems guidance
[23,24], aims to:
NAssess to what extent the need for
health systems guidance is part of
national policies and plans and how
guidance is currently formulated by
analyzing strategic health sector docu-
ments from LMICs;
NDescribe the methodological challeng-
es in outlining the approaches to
produce health systems guidance and
to suggest ways to address these
challenges.
The second article in this series ex-
plores the challenge of linking guidance
development and policy development at
global and national levels and examines
the range of factors that can influence
policy development [23], and the third
article explores the challenge of assessing
how much confidence to place in evi-
dence on health systems interventions
[24].
The Need for Health Systems
Guidance in LMICs
To assess the need for guidance and
how it is formulated, we scrutinised the use
of guidance-related terms in national
Summary Points
NWeak health systems hinder the implementation of effective interventions;
policies to strengthen such systems need to draw on the best available
evidence.
NHealth systems evidence is best delivered in the form of guidance embedded in
policy formulation processes, but health systems guidance is poorly developed
at present.
NThe translation of research on problems, interventions, and implementation
into decisions and policies that affect how systems are organised is one
challenge facing the development of health systems guidance.
NThe development of guidance that is timely and usable by the broad range of
health systems stakeholders, and of methods to appraise the quality of health
systems guidance, are additional challenges.
NFurther research is needed to adapt existing approaches (e.g., those used in
clinical guidelines) to produce meaningful advice that accounts for the
complexity of health systems, political systems, and contexts.
NThis is the first paper in a three-part series in PLoS Medicine on health systems
guidance.
Box 1. Health Systems Guidance and Knowledge Translation
Evidence-informed health systems guidance tackles health systems problems by:
NFraming health systems problems;
NSystematically retrieving, translating, and packaging the best available evidence
on health systems interventions and implementation issues;
NUsing this evidence to recommend and formulate—in a deliberative process—
options to solve these problems and to inform policy-making, the level of
decision-making where different courses of action are considered;
NProviding insights on the strategies that can be followed in order to implement
and evaluate a given health systems policy.
Guidance needs to be transparent, systematic, and adapted to the local contexts.
It therefore needs to use validated approaches, to consider all the available
evidence and to assess its quality. It also needs to take into account local factors
that may influence the effects of all options recommended and to address their
feasibility.
Several global institutions have legitimate roles in producing guidance, often in
response to requests from national decision makers (see the second paper in this
series [23]). Thus, to avoid unnecessary duplication of the enormous efforts of
guidance development, coordination is needed at the global level. However, as
decisions on the options recommended are taken at national level, global
guidance needs adaptation, ideally through national deliberative processes.
The production of guidance is largely based on knowledge translation
approaches that bridge the gap between research evidence and its application
to policy-making [53,54]. Figure 1 schematically represents these approaches
across the research, policy, managerial, and societal domains.
PLoS Medicine | www.plosmedicine.org 2 March 2012 | Volume 9 | Issue 3 | e1001185
health policy and strategic documents
from LMICs (Box 2). We found that the
terms ‘‘guidance’’ or ‘‘guidelines’’ fre-
quently appeared in strategic documents
but were more often related to clinical
matters than to health systems (Box 3).
Challenges in Outlining the
Approaches to Produce Health
Systems Guidance
To improve how WHO responds to
requests for guidance on health systems
and the quality of the guidance produced,
WHO recently commissioned the produc-
tion of a Handbook outlining approaches
to develop health systems guidance [25]. A
Task Force on Developing Health Systems
Guidance (listed in the Acknowledgments
section) was formed from WHO expert
panels and from experts working in the
field of health systems research at interna-
tional and academic institutions and
oversaw the drafting of the Handbook for
Developing Health Systems Guidance: Supporting
Informed Judgements for Health Systems Policies
[25] (which was based on existing best
practices for the development of clinical
guidelines, on approaches and tools de-
scribed in the literature, and on the
expertise of the Task Force members) by
reviewing and commenting on its content
by email and through regular teleconfer-
ences and meetings in person. During the
production of the Handbook, because
evidence from research (for example, on
the effectiveness and acceptability of
health systems interventions) is not guid-
ance and is insufficient for optimal deci-
sion-making [26], several conceptual and
methodological challenges associated with
the production of health systems guidance
were identified. The writing group for this
paper further considered these issues and
produced this manuscript, which was
finalised after several iterations of com-
ments by the Task Force and external
reviewers. Here, we discuss four specific
challenges that were encountered during
the production of the Handbook, namely:
NResearch on effectiveness is typically
articulated around health interven-
tions, but policy decisions often relate
to arrangements of the health system,
services, and programmes and encom-
pass multiple interventions packaged
into a particular policy;
NUsers and producers of guidance
include a broad range of stakeholders
who may not all be familiar with
research methods;
NThe production of guidance has to be
timely in relation to the need and
available capacity to implement it and
consistent with countries’ priorities;
NThe quality of guidance needs to be
appraised using transparent criteria;
guidance then needs to be disseminat-
ed and promoted actively to facilitate
its uptake.
Translating Research on Problems,
Interventions, and Implementation
into Decisions on Policies and
Services
Many processes, frequently involving
iterations from research evidence into
policy formulation and from policy evalu-
ation into research prioritisation, are need-
ed to bridge the gaps between research,
policy, and practice. The complexity of
these processes demands a dynamic frame-
work [27] that is comprehensive and
incorporates current thinking about evi-
dence, policy formulation, and health
systems such as the one shown in Figure 1.
Research is articulated around research
questions that can be addressed using
particular methods and is driven by re-
search opportunities (e.g., funding), re-
searchers’ interests, and feasibility. By
contrast, policy decisions and the manage-
rial arrangements needed to put them into
practice have to be responsive to population
needs and integrated into complex health
systems that go beyond the mere aggrega-
tion of single interventions. For example,
the research question of whether lay health
workers are effective in delivering specific
health care interventions will become fully
relevant for policy only when the essential
components of the intervention (e.g., train-
ing of lay health workers), related actions
(e.g., adaptations needed in the distributions
of tasks of cadres), implementation issues
(e.g., the preferences of potential clients),
and the implications across other health
system building blocks (e.g., adaptations to
the health information sub-system that may
be needed to capture the tasks undertaken
by such workers) are all considered.
Ideally, implementation and contextual
issues should be considered when conduct-
ing the systematic reviews of effectiveness
that are needed to translate research into
health systems guidance or in other types
of reviews addressing them [28,29]. Al-
though there are frameworks to analyse
barriers to implementation [30] and the
applicability of evidence [31], many sys-
tematic reviews fail to consider issues
pertaining to applicability and equity, or
the complexity of interventions in relation
Box 2. Assessing Demand for Health Systems Guidance:
Methods
To assess the need of guidance and how it is formulated:
NTwo authors examined all available documents written in English and French in
the Country Planning Cycle Database [55], which has gathered documents that
describe national health policies or strategies since 2005, for the following
LMICs: Afghanistan, Bangladesh, Benin, Burkina Faso, Burundi, Cambodia,
Central African Republic, Chad, Comoros, Congo DR, Eritrea, Ethiopia, The
Gambia, Ghana, Guinea, Guinea Bissau, Haiti, Kenya, Korea DPR, Kyrgyzstan, Lao
People’s Democratic Republic, Liberia, Madagascar, Malawi, Mali, Mauritania,
Myanmar, Nepal, Niger, Rwanda, Sierra Leone, Solomon Islands, Somalia,
Tanzania, Togo, Uganda, Zambia, and Zimbabwe.
NThe documents were searched using the terms ‘‘guideline(s)’’, ‘‘guidance’’,
‘‘guide(s)’’, ‘‘recommendation(s)’’, and their French equivalents and sentences or
paragraphs where the terms appeared were extracted.
NOne of the authors classified the statements extracted from the documents
according to the area of interest (e.g., clinical, public health, health systems),
health system component, the purpose of guidance (e.g., for setting standards,
to guide policy decisions), the decisional scope of guidance (e.g., national, sub-
national), guidance developers and sources (e.g., Ministries of Health, donor,
United Nations agencies), guidance production, and the topics of guidance
according to the WHO Health Systems Framework [5] (SI1).
NOf the 195 documents retrieved from the database, 157 dealing with specific
programmes or strategies were excluded.
NOf the 661 statements retrieved in the 38 remaining documents, 161 were
excluded because the term ‘‘guidance’’ was not used in the sense of decision-
support (relevant for our analysis), but rather as generic advice or because the
contexts where the terms appeared did not fit into any of the areas described
above.
PLoS Medicine | www.plosmedicine.org 3 March 2012 | Volume 9 | Issue 3 | e1001185
to their technical feasibility [32]. This is
partially due to the lack of this type of
information in the primary research that
underlies systematic reviews. Where this
evidence is missing or scarce, efforts to
search, appraise, and synthesise additional
evidence on implementation issues will
have to be made. It will also be important
to consider how to address inequities in
coverage of health care [30], which are
pervasive in many countries.
The production of guidance not only
entails putting research findings into
context, but also entails taking a health
systems perspective. New trends in health
systems thinking advocate the construction
of conceptual pathways that look at the
interventions from a systems perspective,
making explicit how interventions may
trigger reactions in related components of
the health system that may produce
unintended consequences [33,34]. For
example, increasing the salary of health
workers in HIV/AIDS treatment pro-
grammes may result in a reduction of the
health workforce addressing other condi-
tions who do not receive the same salary
benefits.
Finally, it is important to note that the
study designs for research (and the meth-
ods used to synthesise research findings)
that explore different health systems com-
ponents vary. For example, research
exploring the governance and leadership
components of health systems is likely to
involve qualitative methods, whereas re-
search on health care delivery could utilise
both qualitative and quantitative methods
(see also the third paper in this series [24]).
Qualitative synthesis methods have ad-
dressed the nature of problems such as
patient adherence to treatment for tuber-
culosis [35]. Syntheses of mixed method
research can address policy-oriented ques-
tions using framework synthesis, a method
that has been applied to qualitative and
mixed methods research addressing re-
search management [36], public health
[37,38], and workforce management [39].
Producers and Users of Guidance
Although research evidence to inform
guidance is generated through research
synthesis (i.e., systematic reviews) and
often uses complex statistical methods,
guidance is typically produced through a
deliberative process where evidence is
interpreted and contextualised (Figure 1,
Policy domain). During the deliberative
process, which can include a wide range of
stakeholders, potential users, and benefi-
ciaries of guidance, knowledge is ex-
changed, filtered (stakeholders decide on
the relevance of evidence), and amplified
(stakeholders stress evidence consistent
with their views). Colloquial (non-re-
search) evidence (e.g., tacit or experiential
knowledge) helps to interpret or contex-
tualise research evidence and addresses
issues for which research evidence is not
available but that may play a role in the
decisions (e.g., considerations about the
political implications of decisions, or about
potential vested interests of stakeholders)
[40]. Making colloquial evidence explicit
adds transparency to the guidance devel-
opment process.
Box 3. Assessing the Demand for Guidance: Findings
The scope of guidance could be discerned in 63 statements from the included
documents:
N35 statements (56%) were national (i.e., supported decisions or activities
implemented consistently across a country)
N14 (22%) were sub-national
N4 (6%) referred to international guidance
N20 (32%) related to stakeholders, certain types of health facilities, or all levels of
care.
The area of guidance could be identified in 407 statements:
N201 statements (49.3%) referred to clinical issues
N63 (15%) referred to public health issues
N143 referred to miscellaneous areas (e.g., laboratory, management).
In 283 statements (either related to clinical, public health, or miscellaneous
issues), references to one or more WHO health systems building blocks [5]
included:
N83 (29%) on governance and leadership (e.g., roles of governing bodies in
producing or implementing guidance)
N20 (7%) on financing (e.g., drug revolving funds)
N53 (19%) on health workforce (e.g., training)
N67 (24%) on medical supplies (including traditional medicines)
N31 (11%) on information systems (e.g., data for measuring performance)
N29 (10%) on service delivery (e.g., basic health care package)
Documents from eight countries had explicit statements suggesting guidance as
a strategy to improve the ‘‘quality of care’’. Guidance statements related to
traditional medicine were found in The Gambia, Liberia, Rwanda, and Zimbabwe.
The main bodies developing guidance were international organisations (e.g.,
WHO), ministries of health, and special national committees usually linked to the
ministries of health. Guidance production and use was mainly referred to in
connection with high-level health sector entities, or at the sub-national level in
decentralised structures, although similar terms were used in relation to
implementation strategies or operational instructions. There were also statements
emphasising the need for producing guidance, for making it accessible to users,
and for reinforcing adherence. Finally, some documents explicitly related the
development of guidance to the concepts of evidence-based health care.
Our search had several limitations. First, we may have missed some relevant
documents. Second, statements do not necessarily reflect the status that
governments or stakeholders ascribe to guidance. Finally, the terms used to
refer to health systems guidance can differ between countries and across
languages. Our findings nevertheless show that the term ‘‘guidance’’ appeared
frequently in strategic documents. In its ‘‘technical’’ meaning (i.e., systematically
developed statements to assist decisions), the term seems to be more often
related to the clinical field rather than to health systems. This could be because
the concept of formal guidance applied to health systems decisions is not yet
well-established among policy makers or because there are only few examples of
such guidance to draw from. However, the statements linking guidance in a more
generic sense to health systems actions were plentiful in relation to the functions
typically ascribed to ministries of health.
PLoS Medicine | www.plosmedicine.org 4 March 2012 | Volume 9 | Issue 3 | e1001185
Typically, users and producers of guid-
ance, who may include economists, man-
agers, administrators, social scientists and
other professional groups as well as elected
politicians, come from very different re-
search traditions or none at all and have
very different approaches to research and
decision-making [41]. Therefore, it is
probably unrealistic to expect that they will
all be familiar with issues such as assessing
the quality or appropriateness of systematic
reviews [42]. Furthermore, even in the
ideal situation where policy makers are
familiar with research methods, they may
choose to ignore evidence that creates
uncertainty, questions conventional knowl-
edge, ignores local context [43], or that is
not consistent with pre-determined ideas of
preferred policy options.
Although evidence on the effects of
health systems interventions is relatively
scarce, there are renewed efforts to increase
availability and accessibility of this evidence
[22,44,45]. Such evidence should be pre-
sented in user-friendly formats (e.g., mini-
mising technicalities such as complex statis-
tical outputs) to increase its accessibility
when producing guidance and to facilitate
appropriate interpretation by those without
a strong research background. Finally, the
preferences of different decision makers for
how evidence and guidance should be
presented should be taken into account.
Timeliness in the Production of
Guidance
Timeliness in the production of evi-
dence and guidance is critical, but
systematic reviews can take between one
and two years to complete depending on
the availability of new evidence, and
guidance development can take a year
or more [46]. The bottom of Figure 1 also
shows the potentially long time frames for
the policy decision processes and the
evaluation of policies. Furthermore, guid-
ance needs to be regularly updated in the
light of new evidence or changes and
modifications to existing, previously ac-
cepted evidence.
Without timely production of research
synthesis and guidance, the gap between
research, policy, and practice cannot be
bridged, even if primary research is
available. Alternative approaches and
methods are therefore needed to reduce
the lengthy time frames needed for
Figure 1. A generic knowledge translation framework across the research, policy, managerial, and societal domains. Vertical
rectangles contain the methods or approaches to bridging each phase, the frames at the bottom indicate the products for each phase, and the
concepts in between the vertical rectangles represent the different forms of knowledge. Systematic reviews are summarised into a unified body of
knowledge that links priority problems with the effects of interventions and implementation strategies. Knowledge summaries support the
deliberative process in which stakeholders develop guidance products that, in turn, result in policies for services and programmes arrangements. The
outcomes of programmes and services are evaluated to ascertain the extent to which the needs of the population have been met. Evaluation should
inform further research, in an iterative process.
doi:10.1371/journal.pmed.1001185.g001
PLoS Medicine | www.plosmedicine.org 5 March 2012 | Volume 9 | Issue 3 | e1001185
producing systematic reviews and for
translating the findings of reviews into
guidance. Approaches that may be useful
include rapid assessment methods [47],
the use of text mining to speed the search
for publications [48], and the adaptation
of existing guidance that can be institu-
tionally endorsed by the organisations
producing or using it [47].
Appraisal, Dissemination, and
Implementation of Guidance
The quality of guidance has to be
assessed to ascertain the extent to which
the guidance used state-of-the-art and
validated methods during its development
and is, therefore, balanced and reliable in
relation to the evidence that informs it. The
AGREE instrument (Appraisal of Guide-
lines Research and Evaluation) and its
recent revision (AGREE II) is used to assess
the quality of clinical guidelines but could
be adapted to assess health systems guid-
ance. The AGREE instrument, which
consists of 23 items, aims to assess the
quality of guidelines, to provide a method-
ological strategy for the development of
guidelines, and to inform guideline devel-
opers on what and how information ought
to be reported in guidelines [49]. Many of
its items are applicable to both clinical
guidelines and health systems guidance, but
some need minor adaptation for the latter.
For example, item 16 in the AGREE II
instrument mentions the ‘‘different options
for the management of the condition or
health issue are clearly presented’’ whereas
in health systems guidance this would need
to be reworded to indicate ‘‘different
options for addressing the health system
topic are clearly presented’’. In addition,
there should be specific reference to the
need for health systems guidance to refer to
the contextual factors that would determine
the extent to which research evidence is
applicable in specific circumstances.
Passive dissemination of guidance will
not ensure its uptake by potential users
[50], particularly if it lacks relevance to
local or national situations. There are
several proven approaches for the effective
dissemination of research evidence and the
promotion of clinical guidelines (e.g., dis-
tribution of educational materials, educa-
tional meetings and outreach visits, involve-
ment of local opinion leaders, audit and
feedback) that have moderate effects on
outcomes [51]. Some of these approaches
may also be relevant to health systems
guidance, although the challenges relating
to dissemination and uptake for these two
types of guidance are different. At a
national level, a relatively small number of
decision makers may need to be influenced
to integrate guidance into national health
policies and plans. Alternatively, sub-na-
tional and local decision makers may be in
a better position to influence guidance
implementation, particularly in more de-
centralised systems. ‘‘Policy briefs’’ and
‘‘policy dialogues’’ are examples of tools
used to improve the contextualisation and
utilisation of guidance [41] (discussed
further in the second article in this series
[23]). Although uptake of guidance de-
pends on how it is presented and dissem-
inated, additional factors may increase the
utilisation of scientific evidence by policy-
makers. These include early, informal
interactions between researchers and policy
makers, the relevance and timeliness of
evidence, and the consistency between the
evidence and the recommended options on
the one hand, and the beliefs, values,
interests, or political goals of policy makers
on the other [52].
Conclusions
We found that high-level policy and
strategic health sector documents frequent-
ly mention the need for the development of
health systems guidance but suggest that
there may still be limited awareness that
health systems guidance can provide key
inputs into policy-making and that the
development of such guidance requires
systematic and transparent approaches,
inspired by the development of clinical
guidelines. However, given the growing
amount of evidence on health systems and
the current initiatives to address methodo-
logical challenges to produce health systems
guidance (e.g., the Handbook for Developing
Health Systems Guidance [25]), the interna-
tional health community at all levels is likely
to be faced with increasing demands for
guidance on health systems issues.
The approaches we identified for pro-
ducing guidance for health systems typical-
ly mirror validated methods developed for
clinical guidelines but with some important
differences. Specifically, to develop health
systems guidance that is relevant and useful
to decision makers, it is necessary to
acknowledge and to find ways of incorpo-
rating the complex interrelations of the
system components, and the numerous
contextual factors that may influence the
effectiveness of interventions, particularly
their effects on disadvantaged populations.
These issues are discussed further in the
second paper in this series [23]. The uptake
of guidance by decision makers and the
additional complexity of the decision-mak-
ing process itself are also important.
Health systems guidance has the poten-
tial to improve decision-making and enable
more efficient use of resources with conse-
quent improvements in the health of
populations. However, such guidance
needs rigorous and transparent processes
of production and evidence-based ap-
proaches to ensure its dissemination and
uptake. These processes and approaches
are still at a rudimentary stage of develop-
ment. Importantly, the availability of health
systems guidance should encourage better
and more comprehensive health systems
research. This, in turn, should foster more
cross-disciplinary approaches to studying
the dynamic interactions within complex
health systems and thus help to develop
new methods to effectively translate health
systems evidence into usable policy guid-
ance that is relevant to stakeholders at
global and national levels, that takes into
account social and ethical imperatives, and
that recognises the complexity of health
systems and political systems within which
such guidance is introduced.
Supporting Information
Alternative Language Summary
Points S1 Translation of the Sum-
mary Points into Spanish by Xavier
Bosch-Capblanch
(DOC)
Alternative Language Summary
Points S2 Translation of the Sum-
mary Points into French Bruno
Clary, William Lenoir, and Lise
Beck
(DOC)
Alternative Language Summary
Points S3 Translation of the Sum-
mary Points into Portuguese by
Bruno Viana
(DOC)
Alternative Language Summary
Points S4 Translation of the Sum-
mary Points into Arabic by Fadi El-
Jardali
(DOC)
Table S1 Characteristics of guid-
ance in health policy or strategy
documents analysed with examples
of quotations
(DOC)
Acknowledgments
We acknowledge other members of the Task
Force on Developing Health Systems Guidance,
who include (with their affiliations at the time
when the Task Force was initiated): Abdul
Ghaffar, Alliance for Health Policy and Systems
Research, World Health Organization (Switzer-
land); Timothy Evans, Assistant Director-Gener-
al, Information, Evidence and Research, World
Health Organization (Switzerland); Davina
PLoS Medicine | www.plosmedicine.org 6 March 2012 | Volume 9 | Issue 3 | e1001185
Ghersi, Department of Research Policy and
Cooperation, Information, Evidence and Re-
search, World Health Organization (Switzer-
land); Metin Gulmezoglu, UNDP/UNFPA/
WHO/World Bank Special Programme of
Research, Development and Research Training
in Human Reproduction, World Health Organi-
zation (Switzerland); Pierre Ongolo-Zogo, Centre
for the Development of Best Practices in Health,
Yaounde´ Central Hospital, and University of
Yaounde´, Yaounde´ (Cameroon); Ulysses Panis-
set, Department of Research Policy and Cooper-
ation, Information, Evidence and Research,
World Health Organization (Switzerland); Gunn
Vist, Norwegian Knowledge Centre (Norway).
Thanks to Don de Savigny for fruitful
exchanges around health systems and guidance.
Author Contributions
Conceived and designed the experiments: XBC.
Performed the experiments: XBC. Analyzed the
data: XBC. Wrote the first draft of the
manuscript: XBC. Contributed to the writing
of the manuscript: XBC JL SL RA JAR DD LB
AB FEJ LG SO KW PT RK TP AH. ICMJE
criteria for authorship read and met: XBC JL
SL RA JAR DD LB AB FEJ LG SO KW PT
RK TP AH. Agree with manuscript results and
conclusions: XBC JL SL RA JAR DD LB AB
FEJ LG SO KW PT RK TP AH.
References
1. United Nations (2000) Millennium Develop-
ment Goals (MDG). Ref: United Nations Millen-
nium Declaration. United Nations General As-
sembly Resolut ion 55/2. New York: United
Nations. Available: http://www.un.org/millennium/
declaration/ares552e.pdf. Accessed 2 February 2012.
2. UNDP (2010) Millennium Development Goals:
2010 progress chart. Available: http://unstats.un.org/
unsd/mdg/Resources/Static/Products/Progress2010/
MDG_Report_2010_Progress_Chart_En.pdf. Ac-
cessed 30 January 2012.
3. UN (2010) The Millennium Development Goals
Report 2010. Available: http://www.un.org
/millenniumgoals/pdf/MDG%20Report%202010%
20En%20r15%20-low%20res%2020100615%20-.
pdf. Accessed 30 January 2012.
4. Darmstadt GL, Bhutta ZA, Cousens S, Adam T,
Walker N, et al. (2005) Evidence-based, cost-
effective interventions: how many newborn babies
can we save? Lancet 365: 977–988.
5. WHO (2007) Everybody’s business: strengthening
health systems to improve health outcomes:
WHO’s framework for action. Geneva: WHO.
6. Biesma RG, Brugha R, Harmer A, Walsh A,
Spicer N, et al. (2009) The effects of global health
initiatives on country health systems: a review of
the evidence from HIV/AIDS control. Health
Policy Plan 24: 239–252.
7. Travis P, Bennett S, Haines A, Pang T, Bhutta Z,
et al. (2004) Overcoming health-systems con-
straints to achieve the Millennium Development
Goals. Lancet 364: 900–906.
8. Bhutta ZA, Chopra M, Axelson H, Berman P,
Boerma T, et al. (2010) Countdown to 2015
decade report (2000–10): taking stock of mater-
nal, newborn, and child survival. Lancet 375:
2032–2044.
9. Reich MR, Takemi K (2009) G8 and strength-
ening of health systems: follow-up to the Toyako
summit. Lancet 373: 508–515.
10. Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N,
et al. (2010) Health professionals for a new
century: transforming education to strengthen
health systems in an interdependent world.
Lancet 376: 1923–1958.
11. WHO (2010) The world health report - health
systems financing: the path to universal coverage.
Geneva: WHO.
12. Swanson RC, Bongiovanni A, Bradley E,
Murugan V, Sundewall J, et al. (2010) Toward
a consensus on guiding principles for health
systems strengthening. PLoS Med 7: e1000385.
doi:10.1371/journal. pmed.1000385.
13. Sundewall J, Swanson RC, Betigeri A, Sanders D,
Collins TE, et al. (2011) Health-systems strength-
ening: current and future activities (comment).
Lancet 377: 1222–1223.
14. Lewin S, Lavis JN, Oxman AD, et al. (2008)
Supporting the delivery of cost-effective interven-
tions in primary health-care systems in low-
income and middle-income countries: an over-
view of systematic reviews. Lancet 372: 928–939.
15. [No authors listed] (2004) The Mexico Statement
on Health Research. Knowledge for better
health: strengthening health systems. From the
ministerial summit on health research; Mexico
City; 16–20 November 2004.
16.RemmeJHF,AdamT,Becerra-PosadaF,
D’Arcangues C, Devlin M, et al. (2010) Defining
research to improve health systems. PLoS Med 7:
e1001000. doi:10.1371/journal.pmed.1001000.
17. Steering Committee of the First Global Sympo-
sium on Health Systems Research (2010) Mon-
treux Statement from the Steering Committee of
the First Global Symposium on Health Systems
Research. Available: http://www.hsr-symposium.
org/index.php/montreux-statement. Accessed 30
January 2012.
18. Institute of Me dicine (1990) Clinical practice
guidelines: directions for a new program. Wash-
ington (D.C.): National Academy Press.
19. Oxman AD, Fretheim A, Schu¨nemann HJ,
SURE (2006) Improving the use of research
evidence in guideline development: introduction.
Health Res Policy Syst 4: 12.
20. World Health Assembly (24 May 2011) Strength-
ening national policy dialogue to build more
robust health policies, strategies and plans. Sixty-
fourth World Health Assembly WHA64.8. Agen-
da item 13.4. Available: http://apps.who.int/gb/
ebwha/pdf_ files/WHA 64/A64_R8 -en.pdf. Ac-
cessed 30 January 2012.
21. Grimshaw JM, Russell IT (1993) Effect of clinical
guidelines on medical practice: a systematic
review of rigorous evaluations. Lancet 342:
1317–1322.
22. SUPPPORT (n.d.) Supporting Policy relevant
Reviews and Trials. Available: http://www.
support-collaboration.org/index.htm. Accessed
30 January 2012.
23. Lavis JN, Røttingen JA, Bosch-Capblanch X,
Atun R, El-Jardali F, et al. (2012) Guidance for
evidence-informed policies about health systems:
linking guidance development to policy develop-
ment. PLoS Med 9: e1001186. doi:10.1371/
journal.pmed.1001186.
24. Lewin S, Bosch-Capblanch X, Oliver S, Akl EA,
Vist GE, Lavis J , Ghersi D, Røttingen JA,
Steinmann P, Gulmezoglu M, Tugwell P, El-
Jardali F, Haines A (2012) Guidance for evidence-
informed policies about health systems: assessing
how much confidence to place in the research
evidence. PLoS Med 9: e1001187. doi:10.1371/
journal.pmed.1001187.
25. Bosch-Capblanch X and Project Team (2011)
Handbook for developing health systems guid-
ance: supporting informed judgements for health
systems policies. Basel: Swiss Tropical and Public
Health Institute. Available: http://www.swisstph.
ch/?id=53; http://ww w.swisstph.ch/fil eadmin/
user_upload/Pdfs/SCIH/WHOHSG_Handbook_
v04.pdf. Accessed 30 January 2012.
26. Lomas J, Culyer T, McCutcheon C, McAuley L,
Law S (2005) Conceptualising and combining
evidence for health systems guidance. Final
report. Ottawa: Canadian Health Services Re-
search Foundation.
27. Plsek PE, Greenhalgh T (2001) The challenge of
complexity in health care. BMJ 323: 625–628.
28. Bravata DM, McDonald KM, Shojania KG,
Sundaram V, Owens DK (2005) Challenges in
systematic reviews: synthesis of topics related to
the delivery, organization, and financing of health
care. Ann Intern Med 142: 1056–1065.
29. Tugwell P, Petticrew M, Kristjansson E, Welch V,
Ueffing E, et al. (2 010) Assessing equity in
systematic reviews: realising the recommenda-
tions of the Commission on Social Determinants
of Health. BMJ 341: c4739.
30. Fretheim A, Munabi-Babigumira S, Oxman AD,
Lavis JN, Lewin L (2009) SUPPORT tools for
evidence-informed policymaking in health 6:
using research evidence to address how an option
will be implemented. Health Res Policy Syst
7(Suppl 1): S6.
31. Lavis JN, Oxman AD, Souza NM, Lewin S,
Gruen RL, et al. (2009) SUPPORT tools for
evidence-informed health policymaking (STP) 9:
assessing the applicability of the findings of a
systematic review. Health Res Policy Syst 7(Suppl
1): S9.
32. Gericke CA, Kurowski C, Ranson MK, Mills A
(2005) Intervention complexity—a conceptual
framework to inform priority-setting in health.
Bull World Health Organ 83: 285–293.
33. De Savigny D, Adam T, eds. Systems thinking for
health systems strengthening. Geneva: Alliance
for Health Policy and Systems Research.
34. Atun R, Menabde N (2008) Health systems and
systems thinking. In: Coker R, Atun R, McKee M,
eds. Health systems and the challenge of com-
municable diseases: experiences from Europe and
Latin America. Berkshire (United Kingdom):
Open University Press, McGraw Hill Education.
pp 121–140.
35. Munro SA, Lewin SA, Smith HJ, Engel ME,
Fretheim A, et al. (2007) Patient adherence to
tuberculosis treatment: a systematic review of
qualitative research. PLoS Med 4: e238.
doi:10.1371/journal.pmed.0040238.
36. Oliver S, Rees R, Clarke-Jones L, Milne R,
Oakley A, et al. (2008) A multidimensional
conceptual framework for analysing public in-
volvement in health services research. Health
Expect 11: 72–84.
37. Lorenc T, Br unton G, Oliver S, Oliver K,
Oakley A (2008) Attitudes to walking and cycling
among children, young people and parents: a
systematic review. J Epidemiol Community
Health 62: 852–857.
38. Carroll C, Booth A, Cooper K (2011) A worked
example of ‘‘best fit’’ framework synthesis: A
systematic review of views concerning the taking
of some potential chemopreventive agents. BMC
Med Res Methodol 11: 29.
39. Kiwanuka SN, Kinengyere AA, Rutebemberwa E,
Nalwadda C, Ssengooba F, et al. (2011) Dual
practice regulatory mechanisms in the health
sector: a systematic review of approaches and
implementation. London: EPPI-Centre, Social
Science Research Unit, Institute of Education,
University of London, ISBN: 978-1-907345-08-1.
40. Lavis JN, Boyko JA, Oxman AD, Lewin S,
Fretheim A (2009) SUPPORT tools for evidence-
informed health policymaking (STP) 14: organis-
ing and using policy dialogues to support
evidence-informed policymaking. Health Res
Policy Syst 7(Suppl 1): S14.
41. Walshe K, Rundall TG (2001) Evidence-based
management: from theory to practice in health
care. Milbank Q 79: 429–457.
PLoS Medicine | www.plosmedicine.org 7 March 2012 | Volume 9 | Issue 3 | e1001185
42. Lavis JN, Oxman AD, Lewin S, Fretheim A
(2009) SUPPORT tools for evidence-informed
health policymaking (STP). Health Res Policy
Syst 7(Suppl 1): I1.
43. Green A, Bennett S, eds. Sound choices.
Enhancing capacity for evidence-informed health
policy WHO-AHPSR.
44. McMaster University (n.d.) Health systems evi-
dence. Available: http://www.healthsystemsevi-
dence.org. Accessed 30 January 2012.
45. National Library of Med icine (2011) Health
services research (HSR) PubMed queries.
http://www.nlm.nih.gov/nichsr/hedges/search.
html. Accessed 30 January 2012.
46. Raine R, Sanderson C, Black N (2005) Develop-
ing clinical practice guidelines: a challenge to
current methods. BMJ 331: 631–633.
47. UK Civil Service (n.d.) Rapid Evidence Assess-
ment Toolkit. Available: http://www.civilservice.
gov.uk/networks/gsr/resources-and-guidance.
Accessed 30 January 2012.
48. Thomas J, McNaught J, Ananiadou S (2011)
Applications of text mining within systematic
reviews. Research Synthesis Methods 2(1): 1–14.
49. Brouwers M, Kho ME, Browman GP, Burgers JS,
Cluzeau F, et al. (2010) AGREE II: advancing
guideline development, reporting and evaluation
in healthcare. Can Med Assoc J 18: E839–E842.
50. Fretheim A, Schu¨ nemann HJ, Oxman AD (2006)
Improving the use of research evidence in
guideline development: 15. Disseminating and
implementing guidelines. Health Res Policy Syst
4: 27.
51. Grimshaw JM, Thomas RE, MacLennan G,
Fraser C, Ramsay CR, et al. (2004) Effectiveness
and efficiency of guideline dissemination and
implementation strategies. Health Technol Assess
8: iii–iv, 1–72.
52. Lavis JN , Davies H, Oxman A, Denis J-L,
Golden-Biddle K, et al. (2005) Towards system-
atic reviews that inform healthcare management
and policy-making. J Health Serv Res Policy
10(supplement 1): 35–48.
53. Tricco AC, Tetzlaffa J, Moher D (2010) The art
and science of knowledge synthesis. J Clin
Epidemiol 64: 11–20.
54. International Development Centre (IDRC) (n.d.)
What is knowledge translation? Available: http://
web.idrc.ca/en/ev-125530-201-1-DO_TOPIC.
html. Accessed 30 January 2012.
55. International Health Partnership (IHP+) (n.d.)
Country Plan ning Cycle Database: a WHO
resource. Available: http://creator.zoho.com/
downeyc/countryplanningdb/view-perma/
Country_Planning_Cycle_DB. Accessed 30 Jan-
uary 2012.
PLoS Medicine | www.plosmedicine.org 8 March 2012 | Volume 9 | Issue 3 | e1001185
