No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
An online database and desktop assessment software to simplify systematic reviews in environmental science
Abstract
We describe software to facilitate systematic reviews in environmental science. Eco Evidence allows
reviewers to draw strong conclusions from a collection of individually-weak studies. It consists of two
components. An online database stores and shares the atomized findings of previously-published
research. A desktop analysis tool synthesizes this evidence to test causeeeffect hypotheses. The software
produces a standardized report, maximizing transparency and repeatability. We illustrate evidence
extraction and synthesis. Environmental research is hampered by the complexity of natural environments,
and difficulty with performing experiments in such systems. Under these constraints, systematic
syntheses of the rapidly-expanding literature can advance ecological understanding, inform environmental
management, and identify knowledge gaps and priorities for future research. Eco Evidence, and
in particular its online re-usable bank of evidence, reduces the workload involved in systematic reviews.
This is the first systematic review software for environmental science, and opens the way for increased
uptake of this powerful approach.
... Eco Evidence is a freely available framework (Norris et al. 2012), online database of evidence, and supporting software (https://toolkit.ewater.org.au/tools/eco-evidence, accessed 16 February 2018) to facilitate assessment of causal hypotheses in environmental science (Webb et al. 2015b). The history and logic behind the Eco Evidence framework are described in detail elsewhere (Nichols et al. 2011;Norris et al. 2012). ...
... Each study was reviewed for relevant evidence items. Evidence was extracted and was given a weighting based on the study design according to the pre-defined rules in Eco Evidence (Nichols et al. 2011;Norris et al. 2012), and entered into the online Eco Evidence database (Webb et al. 2015b). After extracting evidence items, we assessed the level of support for each cause-effect hypothesis from the conceptual models. ...
... To evaluate support for each hypothesis, we used the Eco Evidence desktop analysis software (Webb et al. 2015b) to sum the individual evidence weights of studies that support the hypothesis, and also sum weights of those that refute it. We used the default threshold of 20 summed points for reaching conclusions. ...
Environmental flows aim to mitigate the impacts of modification of riverine flow regimes by restoring components of the natural flow regime. Explicit evidence-based cause-effect relationships between flow regimes and fish responses are required for defensible flow recommendations. However, flow recommendations are typically based on expert opinion rather than a rigorous test of the available evidence. We used rapid evidence synthesis to systematically review the literature on responses of several native Australian fish to flow-related and other important environmental variables, and tested 13 explicit hypotheses. The hypotheses were related to the condition, reproduction and survival of pygmy perches, carp gudgeons, blackfish and flat-headed gudgeons. The evidence was insufficient to reach strong conclusions for all but three of our hypotheses. Late-spring high flows are associated with increased survival of pygmy perches and carp gudgeons, and exotic fish are associated with decreased survival of these fishes. The evidence that warmer spring water temperatures enhance their reproduction was inconsistent. The dearth of evidence to support or refute most hypotheses points to substantial gaps in our understanding of hydro-ecology of these native Australian fish, highlighting the need for targeted monitoring and research to further understand key flow relationships for these fish species.
... Relevant information from the papers was extracted to a spreadsheet to enable a synthesis of the literature (for convenience it was split into Sheets 2 & 3 in Appendix A). According to the concept, each effect of combined stressors on a response variable (a 'bivariate association' sensu Ziegler et al., 2015) described in a quantitative way forms an 'evidence item' (EI) sensu Webb et al. (2015) that was entered into a separate row of the table. Papers containing more than one EI thus covered several rows of the table. ...
... Here, the knowledge of quantifiable responses to multiple stressors can serve as the basis for risk assessment and appropriate management actions. Decision support systems for causal analysis, such as CADDIS (Norton et al., 2009) and Eco Evidence (Webb et al., 2015 ), use scientific literature to inform evidence-based decision making in environmental management. Both applications rely on literature databases containing information on stressor-response associations reported in the peer-reviewed scientific literature. ...
... 15 alexandra.collins@imperial.ac.uk the accessibility of our research results for environmental management and policy applications. The objective of this initiative is to facilitate inclusion of FWS-published research results in databases that support systematic literature assessments (e.g., Webb et al. 2015, USEPA 2018. Once the research findings are entered into these databases, they can be quickly and repeatedly accessed, increasing the efficiency and timeliness of their application to diverse environmental management questions and tools. ...
... The Eco Evidence software also provides an online database ( Webb et al. 2015), which stores information about causal relationships extracted from environmental science stud-ies. This information is specifically geared to support cause and effect assessments. ...
This deliverable gives an overview of the functionality of the Freshwater Information System (FIS) through a number of screenshots and a short explanatory text. FIS is one of the tools that are developed within the MARS project. It is the web-based information system providing access to information and practical tools generated in MARS. FIS contains informative factsheets for DPSIR, stressors, ecosystem services and MARS case studies on the impact of multiple stressors for ecological status as well as a model selection tool for river basin management. The design and functionality has been discussed internally with MARS partners (Deltares, UDE, IGB) and with end-users during workshops in Delft (September 2015) and Den Helder (October 2016). FIS will be integrated in the Freshwater Information Platform, which aims at bringing together the results of many projects dealing with freshwater ecosystems in one a single platform.
While the assessment of the ecological status of surface water bodies has become quite straightforward nowadays, almost two decades after the WFD has been launched, the inference of appropriate management options from the assessment is still challenging. More precisely, water body managers face the ecological status assessment of a given water body that usually integrates over several or numerous (multiple) stressors impacting the water body. The challenge is to identify the most-impacting stressor(s) and to distinguish them from the minor ones. Such stressor hierarchies are required to infer the appropriate hierarchy of management options to address the relevant stressors. This report presents tools to assist water body managers in the inference of management options to address the impact of multiple stressors on surface water bodies.
The first chapter presents a conceptual model to visualise the published evidence of the impacts of combined stressors (here: nutrient enrichment and fine sediment pollution) on river organisms. The structured evaluation of published evidence can help identify potential interactions of stressors, which then require consideration in water body management.
The second chapter presents an approach to diagnose the causes of deterioration of lowland rivers based on the causes' (stressors') effects on selected diagnostic metrics derived from the macrozoobenthos community. The approach uses a Bayesian (Belief) Network (BN) to statistically infer the probabilities of the causes to be causal for the detected effects at the water body.
In the third chapter, we present an interactive online tool that builds upon the BN as presented in Chapter 2. The tool provides a graphical interface that allows the user to easily enter evidence (i.e. the states of selected effect variables) to the BN. The results are graphically displayed and accompanied by helpful background information and web links to relevant sources of information.
... One such method is Eco Evidence (Norris et al., 2012;Webb et al., 2015). This approach uses an eight-step method that is consistent with the aims and philosophy of systematic review for hypothesis setting, evidence gathering and synthesis, and reporting on conclusions. ...
Failure to address nonflow stressors and flow-mediated connections through catchments is frequently implicated as the cause of disappointing outcomes from experimental environmental water projects. This is a key reason for considering environmental water management as part of an integrated catchment management (ICM) approach, which aligns efforts across government and nongovernment stakeholders to balance protection and use of land, water, and biodiversity. Successful ICM requires an ability to determine the primary stressors affecting environmental function within a catchment. Systematic review of evidence is a practical approach to identification of stressors in river ecosystems and several approaches are available. Implementation of ICM has had mixed results. In Australia, the Victorian catchment management authorities provide a useful case study where environmental water management has been included as part of a broader catchment management program. The Ecosystem Approach provided a set of principles that should be considered in the design of effective ICM in the future.
... Given the proliferation of hydropower dams and other barriers to fish migration in the temperate south, our aim was to assess the evidence for design criteria that would optimise the effectiveness of fishways for non-sport species. We evaluated a total of 19 hypotheses ( Table 1, Table 2) using the Eco Evidence method and software for literature evidence synthesis (Norris et al., 2012;Webb et al., 2015). êTurbine revolution speed éMortality (blade strike) T4 ...
The development of hydropower and other infrastructure that disrupts river connectivity poses a serious threat to highly endemic and genetically distinct freshwater fish species in temperate parts of the Southern Hemisphere. Such locations have been neglected in previous reviews on fish passage. Fishways have long been constructed to mitigate the impacts of riverine barriers on fish, yet they have failed for all but the largest, strongest swimming taxa. This is a particular problem in the temperate south, where the majority of species are small-bodied with low swimming performance. Using the Eco Evidence method for rapid evidence synthesis, we undertook an assessment of evidence for effective fishway design focusing on species representative of the temperate south. Systematic literature searches resulted in 630 publications. Through a rigorous screening process these were reduced to 46 publications containing 76 evidence items across 19 hypotheses relating to design criteria for upstream and downstream passage. Each evidence item was weighted according to the robustness of its study design. These weightings contributed towards the support or rejection of each hypothesis using well-established thresholds. We found an overwhelming lack of evidence for effective fishway design in the temperate south. Particular deficiencies were found with regard to the design of effective facilities for downstream passage. The attraction and entrance of upstream migrating fish into fishways is also relatively under-researched. Given the urgent need for effective fishways in the temperate south, these results justify an approach to fishway design based on a combination of empirical data and expert knowledge. In the meantime, significant resources should be assigned to improve the evidence base through high quality research. The particular deficiencies identified here could guide that research agenda.
... In these cases, the narrow scope restricts the size of the evidence-base, reducing the work involved in all steps of the synthesis process, but can introduce bias. Some rapid review approaches, such as Eco Evidence, have documented methods, use systematic searches, have clear inclusion criteria and weight relevant studies by quality, which provides a robust, yet rapid review of the evidence for causal associations (Norris et al., 2012;Webb et al., 2015b). ...
... Eco Evidence is a synthesis method with an online database and analysis software for use in environmental causal assessments (available from www.toolkit.net.au/tools/eco-evidence). It provides a robust method for synthesis of research findings available in the literature to assess cause-effect hypotheses regarding the effects of human activities or environmental rehabilitation ( Nichols et al. 2011;Norris et al. 2012;Webb et al. 2015b). The method was also designed to help improve the accessibility of primary research to policy-makers and managers. ...
Implementing policies to return water to the environment is challenging. We developed an approach to integrate ecological responses with river management models to provide a link between flow and ecology. This paper aims to summarize the findings of a multifaceted project bringing together multiple lines of evidence to achieve a proof-of-concept approach for optimizing water use to support water policy and management decisions. We developed an eco-hydrology model that represents rivers, water storages, operational constraints, flow management, an ecological response and consumptive irrigation demands, which is linked to climate. To demonstrate the proof of concept, we describe a case study where modelling optimization seeks solutions to minimize terrestrial vegetation encroachment (an ecological response) and maximize irrigation net value through flow management. The solutions provide a range of alternative ‘optimal’ management options to consider. Ultimately, hydrological modelling that integrates both consumptive use and ecological response simultaneously is important to enhance stakeholder understanding of the complexities and trade-offs involved in implementing different decisions. Although proof of concept at this stage, the approaches offer potential for integration of river management models and quantitative ecological response models (derived by considering available evidence from many sources) to support water policy and management decisions.
... The need for tools and a standardized approach to environmental causal assessments were the primary motivations for developing Eco Evidence (www.toolkit.net.au/tools/eco -evidence), which comprises an evidence synthesis method (Nichols et al. 2011, an online database, and analysis software (Webb et al. 2015). The goal was to provide a robust and streamlined method and associated tools for systematic literature review to assess cause-effect hypotheses regarding the effects of human activities and rehabilitation on the environment , Nichols et al. 2011. ...
Investigators attempting to demonstrate causal relationships in environmental systems face challenges associated with natural variability, multiple stressors, the difficulty of performing rigorous experiments, and the time and money required to undertake such studies. Rapid, transparent, and logical methods are needed to synthesize and evaluate evidence from multiple existing scientific research studies. Eco Evidence was developed for this purpose. It provides a synthesis method, online database, and analysis software for use in environmental causal assessments. Eco Evidence provides a definition of 'acceptable' and 'sufficient evidence' to establish causation. To illustrate its capabilities, we provide an example in which Eco Evidence informed a review of groundwater abstraction limits for environmental flow guidelines in the Australian Capital Territory. The synthesis of available evidence from peer-reviewed and unpublished sources was deemed sufficient and acceptable to inform a decision to keep groundwater abstraction limits <10% of long-term recharge. However, the definitions of acceptable and sufficient evidence might differ based upon the reasons for conducting an environmental causal assessment and on the perceived risk and consequences of making particular decisions.
... location, study design, level of replication, effect size, quality and strength of the dependence) further aid in interpreting and weighting individual associations in the context of new hypotheses and analyses. The extraction of such information from Mims and Olden (2012) is detailed in Webb et al. (2015). We are developing an open-access, online and machinereadable repository of associations, external, but complementary, to the traditional written manuscript and scientific publication paradigm. ...
Responsible care and management of Earth’s resources requires scientific support, but the pool of underused
research is growing rapidly. Environmental science research studies describe associations between
variables (e.g. statistical relationships between stressors and responses). We propose open-access and
online sharing of such associations. This concept differs from various efforts around the world to promote
sharing of primary research data, but holds similar goals of improved use of existing knowledge. The initiative
is made possible by recent developments in information technology and evolving online culture
(e.g. crowdsourcing and citizen science). We have begun to connect existing projects that catalog and
store associations, thereby moving toward a single virtual repository. Researchers and decision makers
may share and re-use associations for myriad purposes, including: increasing efficiency and timeliness
of systematic reviews, environmental assessments and meta-analyses, identifying knowledge gaps and
research opportunities, providing evolved metrics of research impact, and demonstrating connections
between research and environmental improvement.
The place of political participation in the democratization process in Africa necessitates understanding the motivation for participation as an integral component for democracy. For Nigeria and Africa, this is crucial for the emergence of good governance and the development of democratic processes and governance. This chapter explored political participation in Nigeria and the issues of political apathy as a challenge of the democratization process. Using a desktop review technique, the chapter explored the causes of political apathy and its possible effect on Nigeria's democratization process as a template for Africa. While the factors and impacts identified in this chapter for political apathy in Nigeria are not exhaustive, the implication for the democratization process is significant. Therefore, there is a need for all stakeholders, governments, and citizens alike to embrace proactive measures in curtailing political apathy in Nigeria and, by inference, Africa's democratization process.
Globally, many river systems are under stress due to overconsumption of water. Governments have responded with programmes to deliver environmental water to improve environmental outcomes. Although such programmes are essential, they may not be sufficient to achieve all desired environmental outcomes. The benefits of environmental water allocation may be improved using ‘complementary measures’, which are non-flow-based actions, such as infrastructure works, vegetation management and pest control. The value of complementary measures is recognised globally, but their ecological benefits are rarely well understood, either because there is limited experience with their application, or the importance of context- and location-specific factors make it difficult to generalise benefits. In this study, we developed an approach to evaluate complementary measures at different levels of detail as a mechanism to aid decision-making. For systems that require a rapid, high-level evaluation, we propose a score-based multi-criteria benefit assessment module. If more ecological detail is necessary, we outline a method based on conceptual models, expert elicitation and probability assessment. These results are used to populate a cumulative benefit assessment tool. The tool evaluates the benefits of proposed measures in the wider context by including variables such as flow, dependence on ongoing maintenance and additional ecological values. We illustrate our approach through application to the Murray–Darling Basin, Australia. As many water recovery programmes mature into their evaluation phases, there is an increasing need to evaluate the ecological benefits of including complementary measures in the toolkit available to policy makers.
Sargassum algal blooms on ocean surfaces and landings of huge Sargassum mats on beaches is an emerging global environmental challenge with wide socio-economic and environmental implications. Literature on Sargassum growth cycles, travel patterns, species and morphotypes, and quantified impacts have tended to focus on a geographic region, or a specific event. Few, if any, publications document long term continuous monitoring of Sargassum blooms in large areas such as the Pacific, or the tropical Atlantic. To address this gap, this paper systematically reviews the global evidence of Sargassum bloom monitoring beyond the Sargasso Sea, and identifies gaps in the evidence base of floating and landing influxes. This systematic review uses combinations of two key terms relating to Sargassum and monitoring, and utilises the resources in ISI Web of Knowledge, Scopus and Google Scholar. The analysis moves us past a classic literature review, and produces an unbiased assessment of empirical research on Sargassum monitoring from 1960 to 2019. We find a significant research focus on open-ocean blooms and floating mats whereas research on beach landings and their associated impacts is comparatively limited. Research is focused within specific countries or water bodies (notably, the Gulf of Mexico, the Caribbean and North Atlantic Ocean) and tends not to comprehensively assess neighbouring or regional shorelines, for example, West Africa and Central America. There was a lack of consistency in the application of methods for quantifying Sargassum biomass volume (including dry/wet weight, unit of measurement, and spatial extent of calculations). Further, in many publications Sargassum species identification was omitted. Given current attempts to understand the drivers and impacts of the exponential growth in Sargassum in some parts of the world, a consistent and replicable research approach to monitoring Sargassum could support creation of a Sargassum evidence database. To move this agenda forwards, we propose a definition for a Sargassum 'event': a continuous bloom of any Sargassum in open oceans, or, an aggregation of landed Sargassum, with the potential to disrupt social, economic or ecosystem functioning, or to impact human health. This review highlights the importance of standardising Sargassum monitoring methods to facilitate improved documentation of temporal and spatial patterns of Sargassum blooms and beach landings.
The diffuse nature of nonpoint source (NPS) pollution as well as that of the effectiveness of best management practices (BMP) to control NPS pollution necessitates BMP evaluation at the field scale. In this study, a web interface was developed for application of the Agricultural Policy Environmental eXtender (APEX) model at the field scale. The interface contains background databases for field location, soil, agricultural management and climate across the contiguous United States. Users can specify properties to run the model for an individual field and compare the results under various land management and conservation practice choices. A case study was conducted to demonstrate the capability of the web interface for simulating various land management scenarios. This tool can help provide on-site information for NPS pollution management related policies and serve as a communication tool among scientists, engineers, and stakeholders.
The development of hydropower and other infrastructure that disrupts river connectivity poses a serious threat to highly endemic and genetically distinct freshwater fish species in temperate parts of the Southern Hemisphere. Such locations have been neglected in previous reviews on fish passage. Fishways have long been constructed to mitigate the impacts of riverine barriers on fish, yet they have often failed for all but the largest, strongest swimming taxa. This is a particular problem in the temperate south, which is home to native species that are non-recreational and generally small-bodied with weak swimming abilities (e.g. Galaxiidae) relative to typical target species for fishway design (e.g. Salmonidae). Using the Eco Evidence method for rapid evidence synthesis, we undertook an assessment of evidence for effective fishway design focusing on species representative of the temperate south, including eel and lamprey. Systematic literature searches resulted in 630 publications. Through a rigorous screening process these were reduced to 46 publications containing 76 evidence items across 19 hypotheses relating to design criteria for upstream and downstream passage. We found an overwhelming lack of evidence for effective fishway design in the temperate south. Particular deficiencies were found with regard to the design of effective facilities for downstream passage. The attraction and entrance of upstream migrating fish into fishways is also relatively under-researched. Given the urgent need for effective fishways in the temperate south, these results justify an approach to fishway design based on a combination of empirical data and expert knowledge. In the meantime, significant resources should be assigned to improve the evidence base through high quality research. The particular deficiencies identified here could guide that research agenda.
Objectives:
In the last decade, we saw an upsurge of studies evaluating the role of ecosystem goods and services (EGS) on human health (Eco-Health). Most of this work consists of observational research of intermediate processes and few address the full pathways from ecosystem to EGS to human health, limiting our ability to assess causality.
Methods:
We conducted a causal criteria analysis of Eco-Health literature using Eco-Evidence, a software tool that helps evaluate evidence of cause-effect relationships. We focus on the context of green spaces providing "buffering" EGS that may influence disease.
Results:
We found support for a causal linkage between green spaces and all of the EGS tested, and sufficient evidence linking EGS to gastro intestinal disease and heat morbidities. Inconsistencies were found when assessing the link between EGS to cardiovascular and respiratory diseases. Few studies directly link green spaces to health. Those that do, support a connection to cardiovascular disease, and heat morbidities, but provide inconsistent evidence regarding respiratory illness.
Conclusions:
Our results help establish an agenda to shape future Eco-Health research and define priorities for managing green spaces to provide human health benefits.
Accurate and informative methods for evidence synthesis that are also simple and inexpensive to implement would greatly increase our ability to use scientific research results to better manage natural environments. Across the world, pressure to use evidence-based environmental management is increasing, but little guidance exists as to what 'evidence-based' actually means. Methods for systematic review of literature evidence have been modified from those used in medical research, but the effort involved in using these methods means that they have not been widely adopted. We compared 3 independently developed approaches to 'rapid evidence synthesis' methods developed in an attempt to improve efficiency and responsiveness compared to standard methods for systematic review. Each method has features that should be included in an ideal rapid evidence-synthesis method and has potential for further development. Increasing standardization of methods for evidence extraction, quality assessment, and synthesis increases the transparency and repeatability of the results obtained. However, the most important consideration is that the methods are fit for purpose; i.e., that each method is good enough to do the job required. The methods presented in this BRIDGES cluster, and potentially a combined method derived from them, could reduce the effort and cost of evidence synthesis to the scales required for management decisions.
Hydropeaking has negative effects on aquatic biota, but the causal relationships have not been studied extensively, especially when hydropeaking occurs in combination with other environmental pressures. The available evidence comes mainly from case studies demonstrating river-specific effects of hydropeaking that result in modified microhabitat conditions and lead to declines in fish populations. We used multiple lines of evidence to attempt to strengthen the evidence base for models of ecological response to flow alteration from hydropeaking. First, we synthesized evidence of ecological responses from relevant studies published in the scientific literature. We found considerable evidence of the ecological effects of hydropeaking, but many causal pathways are poorly understood, and we found very little research on the interactive effects of hydropeaking and other pressures. As a 2nd line of evidence, we used results from analyses of large-scale data sets. These results demonstrated the extent to which hydropeaking occurs with other pressures, but did not elucidate individual or interactive effects further. Thus, the multiple lines of evidence complemented each other, but the main result was to identify knowledge gaps regarding hydropeaking and a consequent pressing need for novel approaches, new questions, and new ways of thinking that can fill them.
While the impacts of regulation on river systems have been investigated, the main focus of the research has been on ecological and in-channel impacts. An understanding of the impacts of regulation on floodplain geomorphol-ogy is needed in order to inform management actions, in particular environmental flows. Concurrently with a traditional literature review on the effects of river regulation on floodplain geomorphology, we undertook a systematic review of the evidence using causal criteria analysis. The causal criteria approach tested evidence from multiple studies against a 'checklist' of criteria, constructing an argument for causality by combining a number of pieces of evidence into a single coherent argument. The causal criteria analysis only found support for one hypothesized cause-effect linkage in the conceptual model. In contrast, after discussing many of the conceptual model linkages, the traditional review provided a series of eight hypothesised scenarios to highlight the different possibilities for flood-plain geomorphic change resulting from regulation. The difference was partly as a result of the time taken to enter studies into the Eco Evidence Database, but mainly because of the quality of evidence from geomorphic studies. Low statistical rigour in study designs meant that many papers were inadmissible as evidence. Thus, the causal criteria analysis made transparent the quality of data used in a traditional literature review that might subsequently be used to justify costly management recommendations.
Human‐induced changes in river flow regimes are ubiquitous worldwide. Although numerous case studies have identified negative ecological impacts of changes in different aspects of flow regimes (e.g. magnitude, timing), there have been few attempts to systematically review this literature to derive general relationships regarding ecological responses to changes in flow regimes.
Systematic literature reviews can inform science and management in ecologically complex systems not amenable to experimentation. However, such analysis of existing literature is often limited by inconsistent study design and data reporting. To attempt to overcome these difficulties, we used the recently developed Eco Evidence method and software to analyse 165 studies of ecological responses to changes in river flow regimes.
Eco Evidence provides a rule set and standardised list of terms to assist reviewers to interpret consistently the results of disparate studies. The companion software assists with the synthesis of this information to reach transparent and repeatable conclusions regarding cause–effect hypotheses of ecological responses to environmental drivers.
We compared our results to those of a recent, informal systematic review of the same studies, which is proving extremely influential. Stronger conclusions are reached when evidence is weighted, classified and combined according to the rules in Eco Evidence. Compared to the original review, we reached informative conclusions for a larger number of flow–response hypotheses, found that hypotheses for which the most evidence was available returned inconsistent results, addressed hypotheses at levels of conceptual resolution relevant to management and identified where insufficient evidence exists to reach a conclusion.
Analyses conducted at several levels of conceptual resolution found strong support for many hypotheses regarding ecological impacts. We found a consistent sensitivity to changes in flow regime for both fish and riparian vegetation across a variety of performance metrics. While macroinvertebrate responses varied among performance metrics (e.g. abundance was negatively affected by increases or decreases in flows, diversity was only negatively affected by flow decreases, and assemblage structure was affected by neither), they were largely consistent within these metrics.
We thus conclude that the Eco Evidence approach allowed us to extract more knowledge from the data set than was possible in the original review. Eco Evidence can improve synthesis of the burgeoning ecological literature and improve our general understanding in ecology. Amid widespread calls for ‘evidence‐based’ environmental management, this powerful tool provides managers with a means of using research to help inform complex environmental decision‐making.
http://tinyurl.com/3dhnzmr old in press version on sakai
Sound decision making in environmental research and management requires an understanding of causal relationships between stressors and ecological responses. However, demonstrating cause–effect relationships in natural systems is challenging because of difficulties with natural variability, performing experiments, lack of replication, and the presence of confounding influences. Thus, even the best-designed study may not establish causality. We describe a method that uses evidence available in the extensive published ecological literature to assess support for cause–effect hypotheses in environmental investigations. Our method, called Eco Evidence, is a form of causal criteria analysis—a technique developed by epidemiologists in the 1960s—who faced similar difficulties in attributing causation. The Eco Evidence method is an 8-step process in which the user conducts a systematic review of the evidence for one or more cause–effect hypotheses to assess the level of support for an overall question. In contrast to causal criteria analyses in epidemiology, users of Eco Evidence use a subset of criteria most relevant to environmental investigations and weight each piece of evidence according to its study design. Stronger studies contribute more to the assessment of causality, but weaker evidence is not discarded. This feature is important because environmental evidence is often scarce. The outputs of the analysis are a guide to the strength of evidence for or against the cause–effect hypotheses. They strengthen confidence in the conclusions drawn from that evidence, but cannot ever prove causality. They also indicate situations where knowledge gaps signify insufficient evidence to reach a conclusion. The method is supported by the freely available Eco Evidence software package, which produces a standard report, maximizing the transparency and repeatability of any assessment. Environmental science has lagged behind other disciplines in systematic assessment of evidence to improve research and management. Using the Eco Evidence method, environmental scientists can better use the extensive published literature to guide evidence-based decisions and undertake transparent assessments of ecological cause and effect.
While the impacts of regulation on river systems have been investigated, the main focus of the research has been on ecological and in-channel impacts. An understanding of the impacts of regulation on floodplain geomorphology is needed in order to inform management actions, in particular environmental flows. Concurrently with a traditional literature review on the effects of river regulation on floodplain geomorphology, we undertook a systematic review of the evidence using causal criteria analysis. The causal criteria approach tested evidence from multiple studies against a ‘checklist’ of criteria, constructing an argument for causality by combining a number of pieces of evidence into a single coherent argument. The causal criteria analysis only found support for one hypothesized cause-effect linkage in the conceptual model. In contrast, after discussing many of the conceptual model linkages, the traditional review provided a series of eight hypothesised scenarios to highlight the different possibilities for floodplain geomorphic change resulting from regulation. The difference was partly as a result of the time taken to enter studies into the Eco Evidence Database, but mainly because of the quality of evidence from geomorphic studies. Low statistical rigour in study designs meant that many papers were inadmissible as evidence. Thus, the causal criteria analysis made transparent the quality of data used in a traditional literature review that might subsequently be used to justify costly management recommendations.
This work presents and illustrates the application of hydroPSO, a novel multi-OS and model-independent R package used for model calibration. hydroPSO allows the modeller to perform a standard modelling work flow including, sensitivity analysis, parameter ...
Encroachment of riparian vegetation into regulated river channels exerts control over fluvial processes, channel morphology, and aquatic ecology. Reducing encroachment of terrestrial vegetation is an oft-cited objective of environmental flow recommendations, but there has been no systematic assessment of the evidence for and against the widely-accepted cause-and-effect mechanisms involved. We systematically reviewed the literature to test whether environmental flows can reduce the encroachment of terrestrial vegetation into river channels. We quantified the level of support for five explicit cause-effect hypotheses drawn from a conceptual model of the effects of flow on vegetation. We found that greater inundation, variously expressed as changes in the area, depth, duration, frequency, seasonality, and volume of surface water, generally reduces riparian vegetation abundance in channels, but most studies did not investigate the specific mechanisms causing these changes. Those that did show that increased inundation results in increased mortality, but also increased germination. The evidence was insufficient to determine whether increased inundation decreases reproduction. Our results contribute to hydro-ecological understanding by using the published literature to test for general cause-effect relationships between flow regime and terrestrial vegetation encroachment. Reviews of this nature provide robust support for flow management, and are more defensible than expert judgement-based approaches. Overall, we predict that restoration of more natural flow regimes will reduce encroachment of terrestrial vegetation into regulated river channels, partly through increased mortality. Conversely, infrequent deliveries of environmental flows may actually increase germination and subsequent encroachment.
Electronic supplementary material
The online version of this article (doi:10.1007/s00267-013-0147-0) contains supplementary material, which is available to authorized users.
1. In an effort to develop quantitative relationships between various kinds of flow alteration and ecological responses, we reviewed 165 papers published over the last four decades, with a focus on more recent papers. Our aim was to determine if general relationships could be drawn from disparate case studies in the literature that might inform environmental flows science and management.
2. For all 165 papers we characterised flow alteration in terms of magnitude, frequency, duration, timing and rate of change as reported by the individual studies. Ecological responses were characterised according to taxonomic identity (macroinvertebrates, fish, riparian vegetation) and type of response (abundance, diversity, demographic parameters). A ‘qualitative’ or narrative summary of the reported results strongly corroborated previous, less comprehensive, reviews by documenting strong and variable ecological responses to all types of flow alteration. Of the 165 papers, 152 (92%) reported decreased values for recorded ecological metrics in response to a variety of types of flow alteration, whereas 21 papers (13%) reported increased values.
3. Fifty-five papers had information suitable for quantitative analysis of ecological response to flow alteration. Seventy per cent of these papers reported on alteration in flow magnitude, yielding a total of 65 data points suitable for analysis. The quantitative analysis provided some insight into the relative sensitivities of different ecological groups to alteration in flow magnitudes, but robust statistical relationships were not supported. Macroinvertebrates showed mixed responses to changes in flow magnitude, with abundance and diversity both increasing and decreasing in response to elevated flows and to reduced flows. Fish abundance, diversity and demographic rates consistently declined in response to both elevated and reduced flow magnitude. Riparian vegetation metrics both increased and decreased in response to reduced peak flows, with increases reflecting mostly enhanced non-woody vegetative cover or encroachment into the stream channel.
4. Our analyses do not support the use of the existing global literature to develop general, transferable quantitative relationships between flow alteration and ecological response; however, they do support the inference that flow alteration is associated with ecological change and that the risk of ecological change increases with increasing magnitude of flow alteration.
5. New sampling programs and analyses that target sites across well-defined gradients of flow alteration are needed to quantify ecological response and develop robust and general flow alteration–ecological response relationships. Similarly, the collection of pre- and post-alteration data for new water development programs would significantly add to our basic understanding of ecological responses to flow alteration.
The environment is being increasingly recognized as a legitimate user of water. However, tension between environmental and
consumptive uses remains and environmental water allocations may be subject to legal challenge. Current predictions of ecological
response to altered flow regimes are not sufficiently transparent or robust to withstand such challenges. We review the use
of causal criteria analysis to systematically review ecological responses to changes in flow regimes. Causal criteria analysis
provides a method to assess the evidence for and against cause-effect hypotheses. Relationships supported by sufficient evidence
can inform transparent and robust environmental flow recommendations. The use of causal criteria analysis in environmental
science has been facilitated by the development of the Eco Evidence method and software—a standardized approach for synthesizing
evidence from the scientific literature. Eco Evidence has thus far been used to assess the evidence concerning responses of
vegetation, fish, macroinvertebrates, and floodplain geomorphology to changes in flow regime, and provides a robust and transparent
assessment of this evidence. There is a growing movement internationally to shift from experience-based to evidence-based
methods in environmental science and management. The research presented here is at the leading edge of a fundamental change
in the way environmental scientists use evidence.
KeywordsEnvironmental flows–Environmental management–Environmental policy–Evidence-based practice–Evidence database–Systematic review
The hydrologic regime is regarded as the primary driver of freshwater ecosystems, structuring the physical habitat template, providing connectivity, framing biotic interactions, and ultimately selecting for specific life histories of aquatic organisms. In the present study, we tested ecological theory predicting directional relationships between major dimensions of the flow regime and life history composition of fish assemblages in perennial free-flowing rivers throughout the continental United States. Using long-term discharge records and fish trait and survey data for 109 stream locations, we found that 11 out of 18 relationships (61%) tested between the three life history strategies (opportunistic, periodic, and equilibrium) and six hydrologic metrics (two each describing flow variability, predictability, and seasonality) were statistically significant (P < or = 0.05) according to quantile regression. Our results largely support a priori hypotheses of relationships between specific flow indices and relative prevalence of fish life history strategies, with 82% of all significant relationships observed supporting predictions from life history theory. Specifically, we found that (1) opportunistic strategists were positively related to measures of flow variability and negatively related to predictability and seasonality, (2) periodic strategists were positively related to high flow seasonality and negatively related to variability, and (3) the equilibrium strategists were negatively related to flow variability and positively related to predictability. Our study provides important empirical evidence illustrating the value of using life history theory to understand both the patterns and processes by which fish assemblage structure is shaped by adaptation to natural regimes of variability, predictability, and seasonality of critical flow events over broad biogeographic scales.
We live in interesting times. Portents of impending catastrophe pervade the literature, calling us to action in the face of unmanageable volumes of scientific data. But it isn't so much data generation per se, but the systematic burial of the knowledge embodied in those data that poses the problem: there is so much information available that we simply no longer know what we know, and finding what we want is hard - too hard. The knowledge we seek is often fragmentary and disconnected, spread thinly across thousands of databases and millions of articles in thousands of journals. The intellectual energy required to search this array of data-archives, and the time and money this wastes, has led several researchers to challenge the methods by which we traditionally commit newly acquired facts and knowledge to the scientific record. We present some of these initiatives here - a whirlwind tour of recent projects to transform scholarly publishing paradigms, culminating in Utopia and the Semantic Biochemical Journal experiment. With their promises to provide new ways of interacting with the literature, and new and more powerful tools to access and extract the knowledge sequestered within it, we ask what advances they make and what obstacles to progress still exist? We explore these questions, and, as you read on, we invite you to engage in an experiment with us, a real-time test of a new technology to rescue data from the dormant pages of published documents. We ask you, please, to read the instructions carefully. The time has come: you may turn over your papers...
Real and apparent conflicts between ecosystem and human needs for fresh water are contributing to the emergence of an alternative model for conducting river science around the world. The core of this new paradigm emphasizes the need to forge new partnerships between scientists and other stakeholders where shared ecological goals and river visions are developed, and the need for new experimental approaches to advance scientific understanding at the scales relevant to whole-river management. We identify four key elements required to make this model succeed: existing and planned water projects represent opportunities to conduct ecosystem-scale experiments through controlled river flow manipulations; more cooperative interactions among scientists, managers, and other stakeholders are critical; experimental results must be synthesized across studies to allow broader generalization; and new, innovative funding partnerships are needed to engage scientists and to broadly involve the government, the private sector, and NGOs. Yes Yes
The steps in a systematic review
Step 1: Framing questions for a review
The problems to be addressed by the review should be specified in the form of clear, unambiguous and structured questions before beginning the review work. Once the review questions have been set, modifications to the protocol should be allowed only if alternative ways of defining the populations, interventions, outcomes or study designs become apparent
Step 2: Identifying relevant work
The search for studies should be extensive. Multiple resources (both computerized and printed) should be searched without language restrictions. The study selection criteria should flow directly from the review questions and be specified a priori. Reasons for inclusion and exclusion should be recorded
Step 3: Assessing the quality of studies
Study quality assessment is relevant to every step of a review. Question formulation (Step 1) and study selection criteria (Step 2) should describe the minimum acceptable level of design. Selected studies should be subjected to a more refined quality assessment by use of general critical appraisal guides and design-based quality checklists (Step 3). These detailed quality assessments will be used for exploring heterogeneity and informing decisions regarding suitability of meta-analysis (Step 4). In addition they help in assessing the strength of inferences and making recommendations for future research (Step 5)
Step 4: Summarizing the evidence
Data synthesis consists of tabulation of study characteristics, quality and effects as well as use of statistical methods for exploring differences between studies and combining their effects (meta-analysis). Exploration of heterogeneity and its sources should be planned in advance (Step 3). If an overall meta-analysis cannot be done, subgroup meta-analysis may be feasible
Step 5: Interpreting the findings
The issues highlighted in each of the four steps above should be met. The risk of publication bias and related biases should be explored. Exploration for heterogeneity should help determine whether the overall summary can be trusted, and, if not, the effects observed in high-quality studies should be used for generating inferences. Any recommendations should be graded by reference to the strengths and weaknesses of the evidence
Much of current conservation practice is based upon anecdote and myth rather than upon the systematic appraisal of the evidence, including experience of others who have tackled the same problem. We suggest that this is a major problem for conservationists and requires a rethinking of the manner in which conservation operates. There is an urgent need for mechanisms that review available information and make recommendations to practitioners. We suggest a format for web-based databases that could provide the required information in accessible form.
An increasing number of applied disciplines are utilizing evidence-based frameworks to review and disseminate the effectiveness of management and policy interventions. The rationale is that increased accessibility of the best available evidence will provide a more efficient and less biased platform for decision making. We argue that there are significant benefits for conservation in using such a framework, but the scientific community needs to undertake and disseminate more systematic reviews before the full benefit can be realized. We devised a set of guidelines for undertaking formalized systematic review, based on a health services model. The guideline stages include planning and conducting a review, including protocol formation, search strategy, data inclusion, data extraction, and analysis. Review dissemination is addressed in terms of current developments and future plans for a Web-based open-access library. By the use of case studies we highlight critical modifications to guidelines for protocol formulation, data-quality assessment, data extraction, and data synthesis for conservation and environmental management. Ecological data presented significant but soluble challenges for the systematic review process, particularly in terms of the quantity, accessibility, and diverse quality of available data. In the field of conservation and environmental management there needs to be further engagement of scientists and practitioners to develop and take ownership of an evidence-based framework.
We live in interesting times. Portents of impending catastrophe pervade the literature, calling us to action in the face of unmanageable volumes of scientific data. But it isn't so much data generation per se, but the systematic burial of the knowledge embodied in those data that poses the problem: there is so much information available that we simply no longer know what we know, and finding what we want is hard – too hard. The knowledge we seek is often fragmen tary and disconnected, spread thinly across thousands of databases and millions of articles in thousands of journals. The intellectual energy required to search this array of dataarchives, and the time and money this wastes, has led several researchers to challenge the methods by which we traditionally commit newly acquired facts and knowledge to the scientific record. This has spawned a number of initiatives aiming to uncover this buried knowledge and to transform scholarly publishing paradigms. This article, which has been adapted from our review published in the Biochemical Journal [volume 424 (part 3), pages 317–333], provides an overview of these projects. It culminates with a description of the Semantic Biochemical Journal experiment, an exciting and innovative collaboration with Portland Press Ltd to create Utopia Documents, a new PDFdocumentreader designed to rescue data from the dormant pages of published docu ments. The article you are about to read is, in part, intended as a taster; to get the full, interactive Utopia experience, we encourage you to investigate the full review in volume 424 (part 3) of the Biochemical Journal (www. BiochemJ.org) and the other articles in that issue.
Monitoring Ecological Impacts provides the tools needed by professional ecologists, scientists, engineers, planners and managers to design assessment programs that can reliably monitor, detect and allow management of human impacts on the natural environment. The procedures described are well grounded in inferential logic, and the statistical models needed to analyse complex data are given. Step-by-step guidelines and flow diagrams provide the reader with clear and useable protocols, which can be applied in any region of the world and to a wide range of human impacts. In addition, real examples are used to show how the theory can be put into practice. Although the context of this book is flowing water environments, especially rivers and streams, the advice for designing assessment programs can be applied to any ecosystem.
Responsible care and management of Earth’s resources requires scientific support, but the pool of underused
research is growing rapidly. Environmental science research studies describe associations between
variables (e.g. statistical relationships between stressors and responses). We propose open-access and
online sharing of such associations. This concept differs from various efforts around the world to promote
sharing of primary research data, but holds similar goals of improved use of existing knowledge. The initiative
is made possible by recent developments in information technology and evolving online culture
(e.g. crowdsourcing and citizen science). We have begun to connect existing projects that catalog and
store associations, thereby moving toward a single virtual repository. Researchers and decision makers
may share and re-use associations for myriad purposes, including: increasing efficiency and timeliness
of systematic reviews, environmental assessments and meta-analyses, identifying knowledge gaps and
research opportunities, providing evolved metrics of research impact, and demonstrating connections
between research and environmental improvement.
Justification of a cause-and-effect relationship in environmental impact studies is complicated by inherent qualities of impact data. Lack of randomization and replication invalidate the use of inferential statistics for inferring a causal link, and place special demands on descriptive arguments for causation. Assembly rules for causal arguments have been developed in epidemiology and provide a rigorous structure for descriptive analysis. Explicit use of assembly rules for making causal arguments allows investigators to efficiently organize, study, and present available evidence. Within this framework, statistical tests can be used to determine if populations at study sites were different after the impact, but the establishment of a causal link between the impact and the observed difference is based on an argument. Causal inference by means of argument is consistent with the scientific method of strong inference and increases the likelihood of correct conclusions.
Natural language processing (NLP) is the study of mathematical and computational modeling of various aspects of language and
the development of a wide range of systems. These include spoken language systems that integrate speech and natural language;
cooperative interfaces to databases and knowledge bases that model aspects of human-human interaction; multilingual interfaces;
machine translation; and message-understanding systems, among others. Research in NLP is highly interdisciplinary, involving
concepts in computer science, linguistics, logic, and psychology. NLP has a special role in computer science because many
aspects of the field deal with linguistic features of computation and NLP seeks to model language computationally.
We used the Eco Evidence method for systematic review to analyze the literature for evidence of causal relationships between components of wetland water regimes (waterlogging, inundation, depth, duration, frequency, timing) and their effects on plant establishment, growth, reproduction, assemblage composition and diversity. Using systematic keyword searches of Thompson ISI's Web of Science, we located 102 studies. Some studies examined multiple water regime components and/or plant characteristics. Overall, the papers yielded 281 pieces of evidence relevant to 38 cause–effect hypotheses. Of these papers, 49% studied plant growth responses. Inundation and water depth were the most-commonly studied water regime components (55% of papers each). Very few studies considered flood frequency (4 studies) or timing (8 studies), and in many cases there was insufficient evidence to test the effect of these components on wetland plants. Overall, our results show that increases in the magnitude of water regime components are related to negative effects on plant characteristics. Within this, the strongest set of results related inundation depth to negative impacts on nearly all plant characteristics (we found a positive relation for shoot length). Waterlogging (as opposed to complete inundation) was generally related to improvements in plant characteristics, but there was insufficient evidence to test several hypotheses. A large number of hypotheses (17) showed inconsistent effects of water regime components, reflecting the diversity of responses of different plant taxa, wetland types, and study methods. We were unable to find sufficient evidence to test 10 of the hypotheses, and are confident that these represent true knowledge gaps that can only be filled by future targeted research. Evidence of causal links improves our understanding of the water requirements of wetland plants. Such evidence can inform the development of environmental flow programs to maintain and restore wetland plant communities on regulated rivers.
In this article, I describe a few aspects of natural language processing (NLP), specifically some recent trends. I have chosen these topics as they cover a wide range of current efforts in NLP. They are also sufficiently diverse ranging from new avenues of research in grammars and parsing, statistical approaches to NLP, a relatively new trend in NLP, and integration of language and graphic/animation modalities, a new domain for investigating semantic and pragmatic aspects of language.
1. There is a mismatch between broad holistic questions typically posed in policy formation and narrow reductionist questions that are susceptible to scientific method. This inhibits the two-way flow of information at the science-policy interface and weakens the impact of applied ecology on environmental policy.
2. We investigate the approaches to building policy in the health services as a model to help establish a framework in applied ecology and environmental management by which reductionist science can underpin decision making at the policy level.
3. A comparison of policy documents in the health and environmental sectors reveals many similarities in identifying approaches and specific interventions that might achieve policy objectives. The difference is that in the health services, information on the effectiveness of potential interventions is far more readily available through the collaborative process of systematic review.
4.Synthesis and applications. Decision makers are increasingly looking to produce policies that are shaped by evidence through evidence-based policy making. The approach that we outline here provides a framework for structuring systematic reviews to deliver the evidence on key policy issues in a way that will see a faster return and provide better use of the systematic review methodology in environmental management.
1. Whilst it is widely recognised that a natural flow regime is important for sustaining riverine ecosystems, the relative importance of the various components of flow regime for riparian vegetation dynamics is poorly understood. We sought to determine the current extent of knowledge on the importance of seasonal flow timing for riparian plants by conducting a systematic review of the literature using causal criteria analysis.
2. Using a definition of ‘riparian’ that included riverine, wetland and floodplain systems, we found sufficient evidence to provide strong support for the existence of causal relations between seasonal flow timing and a number of riparian plant processes, namely rates of waterborne dispersal (hydrochory), germination and growth, as well as riparian community composition. There was insufficient evidence to infer a causal relationship between flow timing and the reproduction or survival of riparian plants.
3. Thus, we argue that seasonal flow timing is important for many of the processes that generate and sustain riparian vegetation communities. River regulation, and/or flow management aimed at restoring ecological values, should consider flow timing and its implication for riparian flora. Because of regulation, many of the rivers of south-eastern Australia have inverted seasonal flow patterns. Whilst direct evidence of the effects of this inversion on the flora of these rivers is lacking, the results of our causal analysis allow us to predict how these plant communities may have been affected.
4. However, these predictions must be treated with caution because of the reliance of some of the causal analyses on wetland studies. For riverine flora, further research is particularly needed on the effects of seasonal flow timing on hydrochory, survival and reproduction.
5. Causal criteria analysis provides a defensible and efficient means for assessing the extent of evidence for or against ecological hypotheses of this kind. In this case, systematic review of the literature provided strong evidence to support a number of causal links between seasonal flow timing and riparian vegetation dynamics, whilst also efficiently identifying knowledge gaps.
The environmental movement of the 20(th) century has evolved into a large, diverse and well-financed global community that is increasingly required to prove its worth. Though the environmental sector collects and uses data to determine the status of ecological and social systems, the effectiveness of the programs and policies it uses to affect this status remains largely untested. As governments and donor institutions insist on greater transparency, accountability and evidence of what works and what does not, much is being learned from other fields (e.g. health services, education, international development) and increasingly sophisticated approaches are emerging to manage effectiveness. For example, program evaluation, adaptive management, and systematic review provide frameworks and methods to collect and use information to measure and improve performance. However, the critical data and collaborations necessary for an effectiveness revolution are marginalized by technical, cultural and political obstacles. Learning from other fields, the environmental sector must exploit key leverage points, such as flows of information and self-organization, to overcome impediments and create incentives to initiate and realize an era of effectiveness in environmental management.
Computerized clinical decision support (CDS) aims to aid decision making of health care providers and the public by providing easily accessible health-related information at the point and time it is needed. natural language processing (NLP) is instrumental in using free-text information to drive CDS, representing clinical knowledge and CDS interventions in standardized formats, and leveraging clinical narrative. The early innovative NLP research of clinical narrative was followed by a period of stable research conducted at the major clinical centers and a shift of mainstream interest to biomedical NLP. This review primarily focuses on the recently renewed interest in development of fundamental NLP methods and advances in the NLP systems for CDS. The current solutions to challenges posed by distinct sublanguages, intended user groups, and support goals are discussed.
Ecology is inherently cross-disciplinary, drawing together many types of information to address questions about the natural world. Finding and integrating relevant data to assist in these analyses is crucial, but is difficult owing to ambiguous terminology and the lack of sufficient information about datasets. Ontologies provide a formal mechanism for defining terms and their relationships, and can improve the location, interpretation and integration of data based on its inherent meaning. Ontologies have assisted other disciplines (e.g. molecular biology) in unifying and enriching descriptions of data, and ecology can benefit from similar approaches. We review ontology efforts in ecology, and describe how these can benefit research by enhancing the location and interpretation of relevant data for confronting crucial ecological questions.
Advancing ecological research with ontologies What constitutes a good literature review and why does its quality matter? Environ. Model. Softw. 43 Environmental flows can reduce the encroachment of terrestrial vegetation into river channels: a systematic literature review
- J S Madin
- S Bowers
- M P Schildhauer
- M B S C Jones
- M J Stewardson
Madin, J.S., Bowers, S., Schildhauer, M.P., Jones, M.B., 2008. Advancing ecological research with ontologies. Trends Ecol. Evol. 23 (3), 159e168. Maier, H.R., 2013. What constitutes a good literature review and why does its quality matter? Environ. Model. Softw. 43, 3e4. Miller, K.A., Webb, J.A., de Little, S.C., Stewardson, M.J., 2013. Environmental flows can reduce the encroachment of terrestrial vegetation into river channels: a systematic literature review. Environ. Manag. 52 (5), 1201e1212.
Guidelines for Systematic Review in Environmental Management, fourth ed. Centre for Evidence-Based Conservation & Collaboration for Environmental Evidence
- Cebc
CEBC, 2010. Guidelines for Systematic Review in Environmental Management,
fourth ed. Centre for Evidence-Based Conservation & Collaboration for Environmental Evidence, Bangor, Wales.