Figure 1 - uploaded by Saba Hinrichs-Krapels
Content may be subject to copyright.
| Essential inputs and outputs, outcomes and impact of the research process used to explore the 3e's (effectiveness, efficiency and equity) in
Source publication
The allocation of research funding can benefit greatly from robust analysis of what has worked in research. In turn, these analyses can help advocacy initiatives and demonstrate accountability to taxpayers and donors. Capturing and mapping data on the inputs, processes, outputs, outcome and impact of research is crucial for these analyses. In this...
Contexts in source publication
Context 1
... addition to data linking challenges, another challenge to benchmarking impacts for comparisons in efficiency, there are no standardized measures of what constitutes a good impact story, nor which types of research are producing valuable impacts. It has recently been shown that both the general public and researchers value impact in different ways, which calls for the development of future generic measures of "impact utility" using micro-economic approaches such as contingent valuation and discrete choice modeling (Pollit et al., 2016). At present, therefore, a systematic Table 1, is not strictly used, with the exception of studies that benchmark publication and citation per £ spent (Elsevier, 2013). An alternative approach is to measure the rate of return of research, with a view to compare efficiencies of these across funding programmes, disease areas, or countries' investments in research. There are examples where research benefits have been quantified within a disease area, such as cancer ( Glover et al., 2014) and cardiovascular research (Buxton et al., 2008), but limited data availability and the associated necessary assumptions mean that direct comparisons on return on assessment should be ...
Context 2
... the challenge shared with the consideration of "efficiency" in research assessment, is that we do not have systematic and standardized reporting on who benefits and "what works" in research funding (ie the outcomes and impact components of Fig. 1). The "gap maps" from the International Initiative for Impact Evaluation, 3ie, for example, demonstrate what is known and not known from impact evaluations and systematic reviews in particular areas such as education, HIV and AIDS, or agriculture (3IE, 2016). Gathering the data for this, however, can be challenging. For example, the latest edition of Millions Saved by the Center for Global Development has identified cases of proven success in global health (Glassman and Temin, 2016), but this required issuing public calls for submissions of good practice, reviewing systematic reviews databases and conducting interviews with subject matters. Women, for example, may be disadvantaged as the beneficiaries of research, in terms of its health, societal and economic impacts (Sen et al., 2007;Kuhlmann and Annandale, 2015;Schiebinger et al., 2011Schiebinger et al., -2015. There is evidence to suggest that research that does not account for gender differences can result in inaccurate conclusions about how women respond to disease and this in turn will influence the effectiveness of treatment choices ( Bartlett et al., 2005;Johnson et al., ...
Context 3
... studies have reviewed the conceptual tools that have been developed for understanding, assessing and describing research activity (Banzi et al., 2011;Bornmann, 2013;Guthrie et al., 2013;Milat et al., 2015;Greenhalgh et al., 2016). The methods used within these tools include the use of bibliometrics to assess academic impact, quantitative indicators and metrics on economic and health outcomes, qualitative narratives and case studies, and conceptual frameworks such as logic models and related theories of change. All of these require data on the inputs of research, and, depending on the questions asked in the assessment, associated data on outputs, outcomes and impact or a combination of the three. Figure 1 is a simplified illustration of these essential elements of research and how we have con- ceptualized the 3e's in the context of these elements. While we are aware of other conceptual frameworks for describing research processes ( Buxton and Hanney, 1996;CAHS, 2009; Guthrie et al., 2013; Greenhalgh et al., 2016) we are using this simplified illustration to contextualize the 3e's framework. The inputs of research include the funding invested, knowledge brought in, and resources required to deliver the research. The research process includes all the activities that enable the research to happen (ie reviewing of evidence, data collection, analysis, reporting and so forth). Asking if these processes are occurring optimally, or if there is waste, duplication of efforts, or indeed if there is a lack of productivity when comparing across research groups can assess the efficiency of this process, which we explore further below. Research activity leads to outputs, outcomes and wider impact, which can serve to tell us whether research has been effective. Finally, the information on inputs, research processes and outputs, outcomes and impact can all serve to determine if research is equitable. We explore assessment for equity further in the following sections, but emphasize that we interpret this in this context as whether the research achieves specific goals, reaching certain beneficiaries, or addressing specific health ...
Similar publications
![Fig. 1. Various traffic calming measures [1]](profile/Abaid-Ullah/publication/319109147/figure/fig1/AS:527273176829952@1502723129760/Various-traffic-calming-measures-1_Q320.jpg)
![Table 1 Range of observed speeds for different calming measures [26]](profile/Abaid-Ullah/publication/319109147/figure/tbl1/AS:670478507196440@1536865941508/Range-of-observed-speeds-for-different-calming-measures-26_Q320.jpg)
Traffic calming plays a vivacious role for the sustainability of traffic management system. Vertical and horizontal calming devices are used globally to decrease speeds at acceptable levels for the execution of laws with ease. Various types of retarders are used to calm the speeds such as speed breakers, which are installed for the safety of users,...
Rapid urbanization and the gradual disappearance of urban and rural barriers have accelerated rural surplus labor migration. This study focused on the rural household’s livelihood from the perspective of farmland lease-out. Using 382 rural households’ data in Jiangxi Province, we used the seemingly unrelated regression and binary logistic models to...
Citations
... The 3 Es-efficiency, effectiveness, and equityhave been used as critical criteria with which to evaluate the performance of multiple systems, programs, and organizations in diverse contexts. For example, the 3Es have been used to evaluate organizational performance (Davis et al., 2013), assess research impacts (Hinrichs-Krapels & Grant, 2016), monitor the policy development of inclusive education systems (Watkins & Meijer, 2016), and evaluate the system performance of humanitarian logistics location strategies (Liu et al., 2021). ...
A public budget consists of financial input for implementing governmental operations. It is generally expected that financial resources will increase organizational performance because they can be used to obtain other types of resources. This study aims to make a theoretical contribution to the punctuated equilibrium theory literature by examining how the different patterns of budget changes affected the competing dimensions of public service performance (3Es: efficiency, effectiveness, and equity) before and after the Myanmar budget reform in 2011. We collected a two-level dataset including budget allocation data at the ministry level and public service providers’ perceptions of performance at the individual level and employed multilevel modeling. Our results show that the effectiveness of public services can be improved by increasing a significant amount of budget allocation (positive punctuations), but this has no effect on efficiency or equity. However, any level of budget reduction can decrease the efficiency and equity of public services.
... [47] The scope of some research has been narrowed down to the issues such as the study and design of the bibliometric, scientometric and altmetric evaluation criteria and indicators [48][49][50][51][52][53][54][55][56] and research impact measurement. [57][58][59][60][61][62][63][64][65] The purpose of Thelwall and Delgado [66] research was to make an explicit case for the use of data with contextual information as evidence in humanities research evaluations rather than systematic metrics; Data are already used as impact evidence in the arts and humanities, but this practice should become more widespread. Humanities researchers should be encouraged to think creatively about the kinds of data that they may be able to generate in support of the value of their research and should not rely upon standardized metrics. ...
... Although Vanholsbeeck and Lendák-Kabók [64] proposed that effectiveness is related to the concept of accountability, this concept is currently not much considered and dominant in the university evaluation culture; however, in the present study, it has taken a significant part. Hinrichs-Krapels and Grant [60] posed some questions as follows: whether the research has been effective or not? Does research input lead to output, outcome and impact? ...
... A third possible objective, somehow instrumental to the previous ones, can be effectiveness, as not all territories can deliver the expected results (e.g. Hinrichs-Krapels and Grant, 2016). These different objectives can be articulated in multiple instruments, thus opening the challenge of harmoniously pursuing potentially conflicting objectives. ...
... In addition, from the service perspective, efficiency was also defined as a step that was closely related to the cost or economic aspect [35]. This fact proves that the use of proven technology in the medium term will also provide a saving effect in terms of budget, where things that could be done through technology development were data that was no longer relevant would automatically be removed, and then for validation purposes, it was enough to do a field visit based on a sampling technique, where the data visited will be adjusted to the provisions of the applicable regulations. ...
Integrated social services organized through the SLRT (Integrated Referral Service System) by the Puskesos (Social Welfare Center) were a step forward in answering today's increasingly complex social problems. On the other hand, the development of technology and information provided opportunities and challenges to unite the two technologies and integrate social services into a single unit to strengthen further efforts to solve social problems. The involvement of technology through applications was a step forward and quite visionary to take advantage of technological developments, especially the closer technology and information to today's society. Based on these conditions, the purpose of this study was to examine deeply the benefits of using technology in integrated referral services carried out by Puskesos. In line with these objectives, the method used was qualitative with descriptive type. Data collection techniques include documentation studies, observations, in-depth interviews, and focus group discussions. Furthermore, the sampling technique in this study was purposive sampling, and the number of informants in this study amounted to 70, spread over two locations, namely Sleman Regency and Bandung Regency. The results show four benefits that were quite dominant in the use of technology in integrated social services: effectiveness, efficiency, accountability, and public trust. Realizing this benefit, it is hoped that other supporting programs will be involved in the form of socialization related to the use of technology through the use of applications in these services.
... In extreme cases, the established incentives can encourage misconduct by researchers [77] or institutions [78]. • Administrative burden: The potential administrative burden that arises from the conduct of RIA represents a real and significant challenge and consumes resources (time and money) that may otherwise be directed to HMR [14, 51,79]. The displacement of researchers' time due to RIA requirements, also often goes unrecognised and unvalued [76]. ...
Research Impact Assessment (RIA) represents one of a suite of policies intended to improve the impact generated from investment in health and medical research (HMR). Positivist indicator-based approaches to RIA are widely implemented but increasingly criticised as theoretically problematic, unfair, and burdensome. This commentary proposes there are useful outcomes that emerge from the process of applying an indicator-based RIA framework, separate from those encapsulated in the metrics themselves. The aim for this commentary is to demonstrate how the act of conducting an indicator-based approach to RIA can serve to optimise the productive gains from the investment in HMR. Prior research found that the issues regarding RIA are less about the choice of indicators/metrics, and more about the discussions prompted and activities incentivised by the process. This insight provides an opportunity to utilise indicator-based methods to purposely optimise the research impact. An indicator-based RIA framework specifically designed to optimise research impacts should: focus on researchers and the research process, rather than institution-level measures; utilise a project level unit of analysis that provides control to researchers and supports collaboration and accountability; provide for prospective implementation of RIA and the prospective orientation of research; establish a line of sight to the ultimate anticipated beneficiaries and impacts; Include process metrics/indicators to acknowledge interim steps on the pathway to final impacts; integrate ‘next’ users and prioritise the utilisation of research outputs as a critical measure; Integrate and align the incentives for researchers/research projects arising from RIA, with those existing within the prevailing research system; integrate with existing peer-review processes; and, adopt a system-wide approach where incremental improvements in the probability of translation from individual research projects, yields higher impact across the whole funding portfolio.
Optimisation of the impacts from HMR investment represents the primary purpose of Research Impact policy. The process of conducting an indicator-based approach to RIA, which engages the researcher during the inception and planning phase, can directly contribute to this goal through improvements in the probability that an individual project will generate interim impacts. The research project funding process represents a promising forum to integrate this approach within the existing research system.
... Al día de la fecha se carece de estudios que evalúen la efectividad de intervenciones de Enfermería sobre la Calidad de Vida de cuidadores de adultos mayores sobrevivientes de ACV. Según Hinrichs-Krapels y Grant (19) , la eficacia en investigación considera si una intervención propuesta genera el resultado esperado y/o los beneficios o el impacto social. ...
Resumen Objetivo: evaluar el efecto de intervenciones de atención domiciliaria de enfermería sobre la calidad de vida en cuidadores familiares de adultos mayores sobrevivientes de accidentes cerebrovasculares. Método: Ensayo Clínico Aleatorizado, cegado para la evaluación de los desenlaces. Los participantes del estudio fueron 48cuidadores familiares de adultos mayores sobrevivientes de accidentes cerebrovasculares (ACV). El Grupo Intervención recibió tres visitas domiciliarias a cargo de enfermeros un mes después del alta hospitalaria, en las que se les ofreció instrucción relacionada con ACV (es decir, cómo acceder a los servicios de salud y realizar las actividades inherentes a los cuidados) y apoyo emocional. Al Grupo Control se le brindó la orientación habitual de los servicios de salud. La calidad de vida se evaluó mediante el instrumento World Health Organization Quality of Life Assessment (WHOQOL-BREF) y el módulo Old(WHOQOL-OLD) 1semana, 2meses y 1año después del alta. Resultados: en su mayoría, los cuidadores fueron mujeres, hijos o cónyuges. Los cuidadores de los grupos Intervención y Control no presentaron diferencias significativas en términos de su Calidad de Vida general de base. La intervención no ejerció ningún efecto entre la asignación a los grupos y la Calidad de Vida general(p=0,625) con el transcurso del tiempo. Sin embargo, la intervención sí tuvo efecto sobre las Relaciones Sociales (p=0,019) y la Autonomía(p=0,004). Conclusión: la intervención ejerció un efecto estadísticamente significativo sobre la calidad de vida de los cuidadores familiares con respecto a las relaciones sociales y a la autonomía. Registro del ensayo: NCT02807012.
... For this scoping review, the term effectiveness was defined as a measure of the extent to which telehealth services do what they are intended to do for people with aphasia (Hinrichs-Krapels & Grant, 2016). For example, we examined whether the participants' overall communication and/or language abilities improved subsequent to telehealth aphasia intervention (e.g., changes in naming accuracy for expressive language, functional communication skills, severity of language impairment). ...
Purpose
There are increasing demands for aphasia assessment and intervention services to be delivered remotely. The purpose of this scoping review was to address what is known about the delivery of assessments and interventions using telehealth for people with poststroke aphasia. Specifically, the review sought to (a) identify which telehealth assessment protocols have been used, (b) identify which telehealth intervention protocols have been used, and (c) describe evidence on the effectiveness and feasibility of telehealth for people with poststroke aphasia.
Method
A scoping review of the literature published in English since 2013 was conducted by searching MEDLINE, Embase, PsycINFO, CINAHL, and Scopus databases to identify relevant studies. A total of 869 articles were identified. Two reviewers screened records independently, finding 25 articles eligible for inclusion. Data extraction was conducted once and validated by the second reviewer.
Results
Two of the included studies examined telehealth assessment protocols, whereas the remaining studies focused on the delivery of telehealth interventions. The results of the included studies illustrated both effectiveness and feasibility regarding telehealth for people with poststroke aphasia. However, a lack of procedural variation among the studies was found.
Conclusions
Overall, this scoping review yielded continued support for the use of telehealth practices as an alternate mode of delivering both assessment and intervention services to people with poststroke aphasia. However, further research is needed to investigate the range of aphasia assessment and intervention protocols that can be offered via telehealth, such as assessments or interventions that use patient-reported measures or address extralinguistic cognitive abilities.
... have conducted a literature study, where the parameters measuring research performance are from the perspective of the theory of input systems, processes, outputs, productivity and outcomes. Figure 1 explains the summary of several previous studies Alomary, 2020;Brown and Svenson, 1988;Çakır et al., 2015;Dill and Soo, 2005; Hinrichs-Krapels and Grant, 2016;Phillimore, 1989;Savithri and Prathap, 2015;Schroeder and Goldstein, 2016), where aspects and parameters that affect research performance are viewed from the perspective of systems theory. This perspective also provides new insights to map the indicators needed to develop a research performance assessment model based on government policies and realities on the ground. ...
... Productivity describes the level of effectiveness of an organization in achieving its goals by using available resources (Pritchard et al., 2008). Productivity dimensions include output per input Grant and Lewison, 1997;Hinrichs-Krapels and Grant, 2016;Prathap and Ratnavelu, 2015), ratio of PhD student to total students (Marconi and Ritzen, 2015), citation to publication ratio (Avanesova and Shamliyan, 2018;Eskrootchi and Sanee, 2018;Ke et al., 2016;Perry, 2017;Sadeghi-Bazargani et al., 2019;Zharova et al., 2018) and ratio of granted patents to registered patents (McAleer and Slottje, 2005). ...
Purpose
This paper aims to identify and classify the parameters that construct the input, processes, output, productivity and outcome variables that affect performance. These parameters are used in the evaluation model to measure research performance in universities so that they can be used as the basis for making leadership policies both at the national and institutional levels.
Design/methodology/approach
The design of this research is a quantitative research method using a survey questionnaire that was sent to the heads of research institutions at universities in Indonesia. To obtain these parameters, a test for determining the value of the loading factor was used.
Findings
The authors found that input variable parameters consisted of 10 parameters; process variable consisted of 22 parameters; output variable parameters consisted of 8 parameters; productivity variable consisted of 4 parameters; and outcome variable parameters consisted of 10 parameters.
Originality/value
One approach to obtain parameters is through systems theory, where every element that makes up the organization contributes to the achievement of goals. This study attempted to develop parameters in the performance appraisal model of systems theory-based research institutions that are adapted to trends in the direction of research in universities. These parameters are based on aspects of input, process, output, productivity and outcome.
... To the present day, there are no studies assessing the effectiveness of Nursing interventions on Quality of Life in caregivers of aged stroke survivors. According to Hinrichs-Krapels and Grant (19) , effectiveness in research considers whether a proposed intervention produces the expected outcome and/or societal benefits or impact. ...
Objective:
to evaluate the effect of nursing home care interventions on the quality of life in family caregivers of aged stroke survivors.
Method:
a Randomized Clinical Trial, blinded for outcome evaluation. Forty-eighty family caregivers of aged stroke survivors participated in the study. The Intervention Group received three home visits by nurses one month after hospital discharge to provide stroke-related education (i.e., how to access health services and perform care activities) and emotional support. The Control Group received the usual guidance from the health services. Quality of life was assessed using the World Health Organization Quality of Life Assessment (WHOQOL-BREF) instrument and the Old Module(WHOQOL-OLD) 1 week, 2 months, and 1 year after discharge.
Results:
the caregivers were mainly women, children, or spouses. The caregivers in the Intervention Group and Control Group did not significantly differ in terms of their Overall Quality of Life at baseline. There was no interaction effect between group allocation and Overall Quality of Life(p=0.625) over time. However, there was an interaction effect for Social Relations(p=0.019) and Autonomy (p=0.004).
Conclusion:
the intervention exerted a statistically significant effect on the quality of life of family caregivers with respect to social relationships and autonomy.
Trial registration:
NCT02807012.
... 2019;OzoliÚa-Ozola 2011). AngÔu termins efficiency ir viendabÓgs jÁdziens, kur angÔu literat˚r‚ nozÓmÁ to, cik liel‚ mÁr‚ resursu izmantojums noteiktu rezult‚tu ieg˚anai atbilst resursu optim‚lai izmantoanai, lai ieg˚tu noteiktas kvalit‚tes rezult‚tus (Bhagavath 2013;BartueviËienÎ, -akalytÎ 2013;Hinrichs-Krapels, Grant 2016;Roghanian et al. 2012). -Ó definÓcija ietver da˛‚du komponenu: finanu, kvalit‚tes, tehnisko un cilvÁkresursu kopumu, kas, savstarpÁji mijiedarbojoties, raksturo gan ieguldÓjumu, gan rezul-t‚tu. ...
The aim of this article is to reflect the problem of the concept of effectiveness/efficiency, to
promote the understanding of this term and to promote a uniform application of the terminology
in Latvian scientific literature. The concept of effectiveness / efficiency and the interpretation
of the relevant term give rise to discussion in Latvian scientific discourse. The problem of this
topic is linked not only to the understanding of the concept of effectiveness/efficiency, but also
to the terminological classification, the possibilities for assessing the efficiency of certain processes, and the relevance of English terms related to effectiveness / efficiency in the Latvian language.
The different interpretations of the terms efficiency and effectiveness in Latvian scientific
literature influence the quality of economic research by attempting to replace these concepts
with existing terms that already have their place in economic theory. Given the universal use of
the concept of effectiveness / efficiency, the issues related to effectiveness / efficiency assessment
processes are also versatile and non-standardised, including various approaches, structures and
components reflected in scientific literature. To find a prerequisite meaning, theoretical basis,
and main issues relating to effectiveness / efficiency concepts, the authors of this article analyse
these terms in English, Russian and Latvian economic literature, as well as their classifications,
definitions, terminology, and interpretations of the term. The authors conclude that in foreign
literature the English terms efficiency and effectiveness, as well as the related terms are clearly
defined and appropriately positioned. However, they do not have a clear definition and interpretation in Latvian scientific discourse. Despite the translation solutions offered by the Latvian
Academy of Sciences Terminology Commission of the terms efficiency and effectiveness, the
proposed equivalents have not stabilised in Latvian scientific literature and the State Administration documentation.
