Fig 1 - uploaded by Rob William Sanson-Fisher
Content may be subject to copyright.
Hypothetical multiple baseline study design in four communities.
Source publication
The aim of this study was to explore the logistical and methodological strengths and weaknesses of some of the more common research designs which can be used to evaluate the impact of system- or population-level approaches for reducing alcohol-related harms.
This paper identifies studies that have evaluated system or population approaches to reduce...
Context in source publication
Similar publications
If the interest is to calibrate two instruments then the functional relationship models are more appropriate than regression models. Fitting a straight line when both variables are circular and subject to errors has not received much attention.
In this paper, we consider the problem of detecting influential points in two functional relationship mod...
Citations
... Opportunities for more rigorous methodology and design could be explored. However, the tension between the needs of communities and the requirements of rigorous research design need to be considered [46]. ...
Population and strengths-based collaborative approaches to community wellbeing as a vehicle for suicide prevention. Submission made to Prime Minister’s National Suicide Prevention Adviser and the Suicide Prevention Taskforce, December 2019.
... Opportunities for more rigorous methodology and design could be explored. However, the tension between the needs of communities and the requirements of rigorous research design need to be considered [46]. ...
In 2015–2016, the Clarence Valley in Northern New South Wales, Australia, experienced an unexpectedly high number of deaths by suicide, and the resulting distress was exacerbated by unhelpful press coverage. The local response was to adopt a community-wide positive mental health and wellbeing initiative. This paper describes the process and achievements of the initiative called ‘Our Healthy Clarence’. Key stakeholders were interviewed at year two and relevant documents reviewed. Data were analysed using document and thematic analysis. Our Healthy Clarence was established following community consultation, including forums, interviews, surveys and workshops. It adopted a strengths-based approach to suicide prevention, encompassing positive health promotion, primary and secondary prevention activities, advocacy, and cross-sectoral collaboration. A stakeholder group formed to develop and enact a community mental health and wellbeing plan. Factors contributing to its successful implementation included a collective commitment to mental health and wellbeing, clarity of purpose, leadership support from key local partners, a paid independent coordinator, and inclusive and transparent governance. Stakeholders reported increased community agency, collaboration, optimism and willingness to discuss mental health, suicide and help-seeking. Our Healthy Clarence draws ideas from mental health care, community development and public health. This initiative could serve as a model for other communities to address suicide, self-harm and improve wellbeing on a whole-of-community scale.
... If only those intervention components that can be isolated for testing through randomised controlled trials under circumscribed conditions, with the individual as the unit of analysis, can be considered 'quality,' then while we may be certain about the study results, the results tell us little about the effectiveness of the overall population-based intervention. [19][20][21][22][23] The evidence base to support injury prevention at the population level needs to cover community engagement, local contexts, measurement of unintended consequences, feedback loops, the relationship between intervention components, and the connections between area-level and individual-level causes and effects. There is no question that population-level interventions need to be based on the highest-level science. ...
Background Society is the system within which populations exist. Sustained change made at the societal level to reduce population-level indicators of injury morbidity and mortality involves systemic change. Description of problem We will not solve the public health problem of injury simply by educating individuals about the nature of injury risk, improving their risk assessment, and providing these individuals with information to enable them to reduce the level of risk to which they are exposed. Substantial improvement in the societal injury burden will occur only when changes are made at the societal level that focus on reducing the population level indicators of injury related harm. Results In this presentation, we consider a shift from the contemporary systematic approach to injury and violence prevention, to a systemic approach more consistent with the principles of ecological public health. We consider the extent to which the logic of the systematic model, and the related misconceptions about the role of uncertainty in science, limit local, national, and global efforts to minimise injury related harm. The shift from an individual to a population perspective has substantial implications for the way we perceive, direct, undertake, and evaluate injury prevention research and practice. The analogy of “the population as patient” provides a clear illustration of the foundational truths that underpin the preferred public health approach to the prevention of injury. Conclusions We conclude by delineating a new program of work that could be of considerable benefit to the injury-related health of populations.
... If only those intervention components that can be isolated for testing through randomised controlled trials under circumscribed conditions, with the individual as the unit of analysis, can be considered 'quality,' then while we may be certain about the study results, the results tell us little about the effectiveness of the overall population-based intervention. [19][20][21][22][23] The evidence base to support injury prevention at the population level needs to cover community engagement, local contexts, measurement of unintended consequences, feedback loops, the relationship between intervention components, and the connections between area-level and individual-level causes and effects. There is no question that population-level interventions need to be based on the highest-level science. ...
So what does all this mean for the field of injury prevention?
Replacing the systematic approach with a systemic one has three implications for the field of injury prevention. The first for injury research, the second for injury prevention practice, and the third for the role of injury prevention within the institutions that make up our society.
... This design has been used extensively in epidemiology to monitor medication use within states ( Wagner et al., 2002), policy change across communities ( Gilligan et al., 2010), and within the field of health care for evaluating implementation of practice guidelines in different clinical settings (Ramsay et al., 2003). This design, therefore, can be applied across different units of analysis and we propose to use it for testing change in different participation goals in which the effect of the intervention is replicated within each individual and across individuals. ...
ABSTRACT Comparative effectiveness research (CER) strives for evidence to inform clinical decisions for specific clients in typical health care settings. While the randomized controlled trial (RCT) is well-aligned with the objectives of CER, this design may not be feasible or applicable to all research questions, particularly those pertaining to clinical decision making and individually based change. It is important, therefore, to consider alternative approaches, especially when studying complex and subjective outcomes such as children's participation. We propose the use of interrupted time series (ITS) quasi-experimental design for its potential application in determining the effectiveness of participation-focused interventions. In this perspective, ITS design is described, ascertaining its advantages and limitations, and suggestions are provided to overcome challenges to implementation. Results of a case study using the ITS approach to evaluate changes in participation of an adolescent with physical disabilities are described. Finally, strategies to implement this design in practice are suggested.
... Researchers can also examine trends in variables before and after receipt of the intervention. In particular, the trends observed in cohorts that have not yet received the intervention can be useful in understanding the patterns of extraneous variables [63] and [62]. The multiple baseline design has additional strengths, including increased confidence that changes in outcomes are a result of the intervention, if those outcomes are found for each cohort and occur across cohorts with consistent amounts of time elapsed [63]. ...
... In particular, the trends observed in cohorts that have not yet received the intervention can be useful in understanding the patterns of extraneous variables [63] and [62]. The multiple baseline design has additional strengths, including increased confidence that changes in outcomes are a result of the intervention, if those outcomes are found for each cohort and occur across cohorts with consistent amounts of time elapsed [63]. For multiple baseline designs, it is highly recommended that individuals (or population groups) be randomly assigned to different starting points for the intervention [62] and [63], and we followed this recommendation. ...
... The multiple baseline design has additional strengths, including increased confidence that changes in outcomes are a result of the intervention, if those outcomes are found for each cohort and occur across cohorts with consistent amounts of time elapsed [63]. For multiple baseline designs, it is highly recommended that individuals (or population groups) be randomly assigned to different starting points for the intervention [62] and [63], and we followed this recommendation. When random assignment is used, the multiple baseline design offers strong internal validity [62]. ...
Clergy suffer from high rates of obesity, chronic disease, and depression, and simultaneously underestimate the toll these take on their daily functioning. Health interventions are needed for clergy and may be tailored to their occupational context and theological beliefs. Few studies have sought to improve clergy health. No prior studies have utilized a randomized design. Spirited Life is a randomized, multiple baseline study that offered enrollment to nearly all United Methodist Church clergy in North Carolina in fall 2010. A total of 1,114 clergy (response rate=64%) enrolled. Using a multiple baseline design, we randomized participants to three cohorts. Each cohort began the health intervention in one of three consecutive years. The third cohort served as a randomized waitlist control cohort, allowing comparisons between the first and third cohorts. The two-year Spirited Life intervention consists of: 1) a theological underpinning for health stewardship based on incarnation, grace, and response and delivered during workshops; 2) the stress management program Williams LifeSkills; 3) Naturally Slim, an online weight loss program; 4) phone contact with a Wellness Advocate; and 5) $500 small grants for health goals. Metabolic syndrome is the primary endpoint. Stress and depressive severity are secondary endpoints. We measured each construct before, twice during, and at the end of the two-year intervention. Study outcomes, to be published after follow-up data are gathered, will provide evidence of the effectiveness of the combined intervention components of Spirited Life. If successful, the intervention may be considered for use with other clergy and faith populations.
... The primary quantitative methodologies for this evaluation are influenced by recent innovations for evaluating complex interventions implemented at the level of health systems or populations. These include community-based surveys to assess changes in coverage and stigma, facility-based surveys to assess changes in case detection, case studies of district level mental health systems, and studies of cohorts of individuals treated by the mental health care plans, to assess changes in mental health, social, and economic outcomes23242526. All data will be disaggregated by gender, residence (rural/urban), and economic status to monitor equity of access to services and outcomes. ...
The majority of people living with mental disorders in low- and middle-income countries do not receive the treatment that they need.
There is an emerging evidence base for cost-effective interventions, but little is known about how these interventions can be delivered in routine primary and maternal health care settings.
The aim of the Programme for Improving Mental Health Care (PRIME) is to generate evidence on the implementation and scaling up of integrated packages of care for priority mental disorders in primary and maternal health care contexts in Ethiopia, India, Nepal, South Africa, and Uganda.
PRIME is working initially in one district or sub-district in each country, and integrating mental health into primary care at three levels of the health system: the health care organisation, the health facility, and the community.
The programme is utilising the UK Medical Research Council complex interventions framework and the “theory of change” approach, incorporating a variety of qualitative and quantitative methods to evaluate the acceptability, feasibility, and impact of these packages.
PRIME includes a strong emphasis on capacity building and the translation of research findings into policy and practice, with a view to reducing inequities and meeting the needs of vulnerable populations, particularly women and people living in poverty.
... Information from community-based obesity prevention trials and texts providing recommendations regarding their evaluation was synthesised narratively in terms of issues considered to be important in the conduct of rigorous evaluations of community-based interventions, including (13,18,22,23,25,31,32) : ...
... Methodological reviews of previous community-based chronic disease prevention interventions have attributed modest intervention impacts, in part, to high rates of population migration into and out of intervention and comparison communities, or to inconsistent intervention exposure, diluting the intervention effect (13,32) . Such factors also represent a risk to evaluations of community-based obesity prevention programmes. ...
Objective:
The aim of the present study was to review the methodological literature regarding evaluation methods for complex public health interventions broadly and, based on such methods, to critically reflect on the evaluation of contemporary community-based obesity prevention programmes.
Design:
A systematic review of the methods and community-based literature was performed by one reviewer.
Results:
The review identified that there is considerable scope to improve the rigour of community-based obesity prevention programmes through: prospective trial registration; the use of more rigorous research designs, particularly where routine databases including an objective measure of adiposity are available; implementing strategies to quantify and reduce the risk of selective non-participation bias; the development and use of validated instruments to assess intervention impacts; reporting of intervention process and context information; and more comprehensive analyses of trial outcomes.
Conclusions:
To maximise the quality and utility of community-based obesity prevention evaluations, programme implementers and evaluators need to carefully examine the strengths and pitfalls of evaluation decisions and seek to maximise evaluation rigour in the context of political, resource and practical constraints.
... It is recognised that interventions in system-level dimensions, such as those that target access, distribution and policy, will have far-reaching and long-term impacts [2]. Evidence for effective approaches in these dimensions is limited however [2,5], often restricted by their lack of popularity with those who profit from alcohol consumption [6]. ...
... Routinely collected data, such as hospital separations and Medicare data, are used in many fields to study population health issues. Hospital separations [11], drunk driving [12], motor vehicle crash and injury rates [12], and data from police call outs [13] relating to alcohol have been used in descriptive and intervention research, but the data are limited by inconsistent and inaccurate reporting [5]. Routine collection of data relating to alcohol consumption and alcohol-related harm has been shown to be both feasible and efficacious as an intervention strategy through the Alcohol Linking Program [14]. ...
There is a need for evidence to guide alcohol harm reduction programs at the population, system or community level. Such evidence should be derived from methodologically rigorous intervention research. Furthermore, overviews of research output indicate that while interventions are occurring in this field, the dominance of descriptive research continues. Here we present suggestions regarding the most important facilitators of producing high-quality intervention research aimed at reducing alcohol consumption and alcohol-related harm at the population or community level. These suggestions are guided and supported by researchers in the field, whose perceptions were sought through a Web-based survey. Routine collection of relevant data, publication of negative results and reconsideration of funding priorities were ranked highest in terms of their importance in increasing intervention research. The importance of the strategies is marred by limitations of feasibility, clearly acting as a barrier to their adoption. It is likely to be necessary to overcome these limitations in order to achieve change.
Routine Health Information is faced with huge challenges which reduce its decision making and planning yields. The objective of the study was 1) to undertake competency gap analysis of health staff’s performance in routine health information tasks, 2) implement quality improvement process training module, 3) to measure the improvement in the competency level of health staff in performing RHIS tasks after application of Quality Improvement Process principles. A quasi-experimental, uncontrolled before and after study was conducted in 18 health facilities in the Ejisu Juaben Municipal Health Directorate, Ghana. The study involved assessment of competency level and training of 141 staff using Data Quality Improvement Process (DQIP) training module over a twelve-month period. RHIS task competence was measured by a pencil and paper test that measured the ability of respondents to perform calculations, and to interpret and use RHIS results, based on the same indicators as used dimension of confidence level. The study found huge competency gaps among staff amidst high confidence in undertaking RHIS tasks while their performance of these tasks scored objectively, yielded low average scores of improvements in competency gaps; data analysis (-36.9%: +3.6%), data interpretation (-42.2%: +9.8%) and use of data (-44.6%: +2.6%) in the baseline and endline evaluations respectively. The study concludes that Quality Improvement Process (QIP) centred training drives the effectiveness of staff competency in performing RHIS tasks, but not necessarily their confidence level.
