Content uploaded by Peter Fantke
Author content
All content in this area was uploaded by Peter Fantke on Nov 12, 2018
Content may be subject to copyright.
LCIA OF IMPACTS ON HUMAN HEALTH AND ECOSYSTEMS
The USEtox story: a survey of model developer visions
and user requirements
Torbjørn Bochsen Westh &Michael Zwicky Hauschild &
Morten Birkved &Michael Søgaard Jørgensen &
Ralph K. Rosenbaum &Peter Fantke
Received: 10 July 2014 /Accepted: 25 November 2014 / Published online: 4 December 2014
#Springer-Verlag Berlin Heidelberg 2014
Abstract
Purpose USEtox is a scientific consensus model for assessing
human toxicological and ecotoxicological impacts that is
widely used in life cycle assessment (LCA) and other com-
parative assessments. However, how user requirements are
met has never been investigated. To guide future model de-
velopments, we analyzed user expectations and experiences
and compared them with the developers’visions.
Methods We applied qualitative and quantitative data collec-
tion methods including an online questionnaire, semi-
structured user and developer interviews, and review of sci-
entific literature. Questionnaire and interview results were
analyzed in an actor-network perspective in order to under-
stand user needs and to compare these with the developers’
visions. Requirement engineering methods, more specifically
function tree, system context, and activity diagrams, were
iteratively applied and structured to develop specific user
requirements-driven recommendations for setting priorities
in future USEtox development and for discussing general
implications for developing scientific models.
Results and discussion The vision behind USEtox was to
harmonize available data and models for assessing toxicolog-
ical impacts in LCA and to provide global guidance for
practitioners. Model developers show different perceptions
of some underlying aspects including model transparency
and expected user expertise. Users from various sectors and
geographic regions apply USEtox mostly in research and for
consulting. Questionnaire and interview results uncover vari-
ous user requests regarding USEtox usability. Results were
systematically analyzed to translate user requests into recom-
mendations to improve USEtox from a user perspective
and were afterwards applied in the further USEtox
development process.
Conclusions We demonstrate that understanding interactions
between USEtox and its users helps guiding model develop-
ment and dissemination. USEtox-specific recommendations
are to (1) respect the application context for different user
types, (2) provide detailed guidance for interpreting model
and factors, (3) facilitate consistent integration into LCA
software and methods, (4) improve update/testing procedures,
(5) strengthen communication between developers and users,
and (6) extend model scope. By generalizing our recommen-
dations to guide scientific model development in a broader
context, we emphasize to acknowledge different levels of user
expertise to integrate sound revision and update procedures
and to facilitate modularity, data import/export, and incorpo-
ration into relevant software and databases during model
design and development. Our fully documented approach
can inspire performing similar surveys on other LCA-related
tools to consistently analyze user requirements and provide
improvement recommendations based on scientific user anal-
ysis methods.
Keywords Actor-network perspective .Requirement
engineering methods .Toxicity assessment .
User survey .USEtox
Responsible editor: Roland Hischier
T. B. Westh:M. Z. Hauschild :M. Birkved :P. Fantke (*)
Quantitative Sustainability Assessment Division, Department of
Management Engineering, Technical University of Denmark,
Produktionstorvet 424, 2800 Kgs. Lyngby, Denmark
e-mail: pefan@dtu.dk
M. S. Jørgensen
Center for Design, Innovation and Sustainable Transition,
Department of Development and Planning, Aalborg University, A.C.
Meyers Vænge 15, 2450 København, Denmark
R. K. Rosenbaum
Irstea, UMR ITAP, ELSA-PACT—Industrial Chair for
Environmental and Social Sustainability Assessment, 361 rue J.F.
Breton, 5095, 34196 Montpellier, France
Int J Life Cycle Assess (2015) 20:299–310
DOI 10.1007/s11367-014-0829-8
1 Introduction
In life cycle assessment (LCA), several methods, assessment
and modeling tools address the characterization of human
toxicological and ecotoxicological impacts of chemical emis-
sions (European Commission 2010; Hauschild et al. 2011).
However, toxic chemical emissions are still often not or in-
sufficiently characterized in LCA studies. Perceived or actual
differences regarding method or model applicability between
developers and users (including LCA practitioners and deci-
sion makers) might contribute to the lack of addressing toxic-
ity impacts in LCA practice. This is partly because it is
considered fundamentally important that the scientific quality
of models is meeting contemporary standard and represents
state-of-the-art, while less effort is usually put into qualitative
model attributes like usability, maintainability, and interoper-
ability and to meet the requirements of the model users
(Nuseibeh and Easterbrook 2007). To address this gap, we
focus in this study on investigating the visions behind devel-
oping a scientific consensus model for the characterization of
potentially toxic chemical emissions along with investigating
how and why users apply this consensus model in practice. By
comparing the developers’visions with the users’application
practices, we develop recommendations helping to further
align the process of translating model development and im-
provement with user practice. This focus aims at ultimately
supporting an improved and extended application of the tox-
icity characterization of chemical emissions in LCA practice.
1.1 The story of USEtox
Between 1993 and 1999, several consensus-building activities
were conducted by the SETAC
1
Impact Assessment working
groups leading to the definition of a framework for assessing
fate, exposure, and effects of life cycle emissions of toxic
chemicals (Udo de Haes 1996; Udo de Haes et al. 1999b;a;
Udo de Haes et al. 2002). Inspired by this and by previous and
parallel consensus-building activities (e.g., Cowan et al. 1995;
Fenner et al. 2005), the OMNIITOX project was initiated
(Carlson et al. 2004; Guinée et al. 2004; Molander et al.
2004) to develop methods for assessing risks and impacts
associated with chemical emissions from product life cycles.
The project served to develop a common perception of the
field of toxicity characterization modeling and to build the
necessary trust in working together in an efficient way towards
harmonizing existing toxicity characterization models. In
2003, the UNEP
2
/SETAC Life Cycle Initiative (LCI) there-
fore established a Task Force on Toxic Impacts officially
launched in Prague on 22 April 2004 to provide clear guid-
ance for assessing human toxicity, ecosystem toxicity, and
related categories with direct effects on human and ecosystem
health. This task force was largely based on joined forces
between members of the previous efforts and identified that
existing toxicity assessment models only covered a limited
number of substances and that scope, principles, and charac-
terization results varied substantially (Dreyer et al. 2003;Pant
et al. 2004), entailing that many LCA practitioners ignored
toxicity-related impacts in their life cycle impact assessment
(LCIA) step (Hauschild 2005). This led to a process towards
developing a scientific consensus model for the characteriza-
tion of human toxicological and ecotoxicological impacts of
chemical emissions (Fig. 1), starting from four expert review
and framing workshops held between 2003 and 2010
(Aboussouan et al. 2004; Jolliet et al. 2006;McKoneetal.
2006; Diamond et al. 2010) and three model comparison
workshops organized in 2006 (Hauschild 2006b,a;Hauschild
et al. 2006). In these workshops, models were compared based
on investigating a test set of chemicals representing a specific
combination of substance properties and identifying those
processes and factors influencing at least some chemical
groups. This process was oriented in and operated from the
state-of-the-art in various fate, exposure, and effect modeling
communities, the input of which was gathered through the
expert workshops. Result was the development and imple-
mentation of USEtox, a combination of a characterization
factors (CF) database and a model to characterize human
toxicity and ecotoxicity of chemical emissions (Hauschild
et al. 2008; Rosenbaum et al. 2008). More details of the full
consensus-building process including previous and parallel
consensus activities are given in Hauschild et al. (2008), while
a full description of considered factors significantly influenc-
ing characterization modeling for different chemical classes is
provided in Rosenbaum et al. (2008). USEtox was officially
announced on25 May 2010, but version 1.0 was already made
freely available via http://usetox.org on 18 Nov 2009. Model
and factors have since been applied in multiple comparative
impact assessments as further discussed in a special issue of
The International Journal of Life Cycle Assessment dedicated
to USEtox (Hauschild et al. 2011), USEtox characterization
factors have been implemented in LCA software (e.g., GaBi,
SimaPro, OpenLCA, Quantis Suite) and some LCIA methods
(e.g., ILCD LCIA, Impact World+, TRACI 2).
USEtox is continuously being improved and further devel-
oped in international efforts with the aim at meeting current
and future user needs and expectations, facing market devel-
opments and addressing unresolved scientific challenges.
TOX-TRAIN (http://toxtrain.eu), a 4-year EU project (No-
vember 2011 to October 2015), is designed (a) to assess and
develop state-of-the-art tools and data for use in comparative
toxicity assessment that will be proposed to be integrated in
USEtox and (b) to further disseminate USEtox via training
and outreach, via redesigning the official website including
the introduction of a user forum and developing a transparent
1
Society of Environmental Toxicology and Chemistry (http://setac.org)
2
United Nations Environment Programme (http://unep.org)
300 Int J Life Cycle Assess (2015) 20:299–310
update proposal procedure including peer review and via
investigating user requirements and improving the usability
of USEtox in practice. The latter provides the scope of this
study, while further improvement and dissemination activities
are summarized in Fig. 1.
1.2 Assessing user requirements to facilitate further
improvements
To assess USEtox user perspectives and requirements and to
identify different user expectations, we employ state-of-the-art
methods from the field of Science and Technology Studies
including Actor-Network Theory and Requirement Engineer-
ing that focus on users’interactions with science and technol-
ogy (Sismondo 2010). Thereby, a technological element (here:
USEtox) and human actors (here: users) are analyzed as
mutually constituted in a socio-technological network where
they influence each other (Harty 2010). Since humans per-
ceive technology differently and apply it in diverse, changing
contexts, technology developers can never fully anticipate the
ideal product for all users during the development process
(Rohracher 2003). However, assessing user perceptions and
practices can help generate priorities for technology
development. Rohracher (2003) recommends managing tech-
nology development as continuous interaction between devel-
opers and users. Actor-Network Theory and Requirement
Engineering are further detailed elsewhere (Latour 2007;
Nuseibeh and Easterbrook 2007;Sismondo2010). These
methods have already been applied to assess socio-
technological interactions of different software tools (Harvey
2001; Takhteyev 2009;Harty2010) and are suited to focus on
the many relationships between a restricted set of technology
users with the actual technology (Harvey 2001). However, to
our knowledge, such methods have not been applied to de-
velop recommendations for strengthening and improving the
application of environmental assessment models, such as
USEtox. Aiming at analyzing and harmonizing USEtox user
practice and requirements with developer visions in accor-
dance with state-of-the-art methods from stakeholder analysis
and technology development, we focus together with two
Fig. 1 USEtox development timeline including consensus-building pro-
cess between 2003 and 2010 and current improvement and dissemination
activities after 2010.
a
Jolliet et al. (2006),
b
Aboussouan et al. (2004),
c
McKone et al. (2006),
d
Guinée and Hauschild (2005),
e
Hauschild et al.
(2006),
f
Hauschild (2006b),
g
Hauschild (2006a),
h
Diamond et al. (2010),
i
Hauschild et al. (2008), Rosenbaum et al. (2008);
j
Oscarson and
Hauschild (2010),
k
Henderson et al. (2011), Rosenbaum et al. (2011);
l
Hauschild et al. (2011),
m
This study
n
ILCB (2013),
o
OMNIITOX project
(EU FP5 contract: G1RD-CT-2001-00501), Carlson et al. (2004),
Molander et al. (2004);
p
USEtoxPI project (LRI-ACC contract:
MTH1001-01), Jolliet and McKone (2012), Mitchell et al. (2013);
q
TOX-TRAIN project (EU FP7 contract: IAPP-GA-2011-285286),
Bengoa et al. (2014);
r
ExpoDat project initiated by LRI-ACC
ExpoDat2012 workshop, American Chemical Council (2012);
s
QUAN-
TOX project (EU FP7 contract: PCIG14-GA-2013-631910)
Int J Life Cycle Assess (2015) 20:299–310 301
USEtox developers
3
on three objectives: (1) To apply selected
data collection methods. This helps to understand on the one
hand the developers’original vision behind building USEtox.
On the other hand, this helps to understand the aims and
practical experiences of users applying USEtox model and
factors. (2) To categorize and evaluate collected data for
identifying and characterizing the general application trends
of USEtox. Trends are then compared with the developers’
original visions and development perspectives. (3) To use
requirement engineering methods for establishing a set of
specific recommendations to harmonize USEtox developer
aims and user requirements. We then generalize our recom-
mendations in support of an improved consideration of user
requirements when developing and disseminating scientific
modeling tools more generally. Comparing expectations and
experiences of users with developer visions of applying
USEtox will help to improve and extend the application of
the characterization of toxic impacts induced by chemical
emissions in LCA and will further help to guide future model
development based on user requirements.
2Methods
To investigate user requirements and the actor-network rela-
tions around USEtox, we applied a mixed-method design
(Frechtling 2010) including both qualitative and quantitative
data collection methods as shown in Fig. 2.Wecombinedfour
different starting points to collect information about user
practice and the developers’visions and perspectives of de-
veloping USEtox: (1) We analyzed basic statistics over users
that have registered at http://usetox.org and downloaded the
USEtox model and databases. (2) We developed an online
questionnaire, which was disseminated via an LCA forum list
and a list of email addresses collected from users downloading
USEtox. (3) We prepared a set of additional, more detailed
questions and interviewed selected USEtox users and devel-
opers. (4) We extracted additional information about the de-
velopers’visions from relevant peer-reviewed literature. Re-
sults from all four approaches were categorized and evaluated
in an actor-network perspective to identify different user
types. Actual practices of the users were then compared with
the developers’original visions about USEtox users. The
outcome of the evaluation is used to develop and focus spe-
cific recommendations to aid setting priorities in the continu-
ous USEtox development and improvement process and also
discuss general implications for scientific tool development.
The applied methods are detailed in the following.
2.1 Data inputs and questionnaire for assessing general user
practice
With permission from the USEtox development team and
strictly respecting the confidentiality of the data ensured by
inviting two USEtox developers as coauthors, we accessed the
user statistics of their official website containing name, affil-
iation, and country per user. We applied basic statistics to
identify the geographical and sectorial distribution of USEtox
users. On 1 Nov 2011, i.e., within the first 24 months after
model and factors became available online, we counted 551
distinct users registered at http://usetox.org.
An online questionnaire was designed using the online
survey service http://obsurvey.com. Combining the list of
users of the USEtox website with the list of about 2500
individuals
4
registered at the LCA forum (http://pre.nl), an
invitation was sent to more than 3000 potential
respondents. The survey scheme was made accessible for
24 days in November 2011. During this period, 131
responses were received. The questionnaire was
developed to get more detailed and quantitative
information regarding the usage of USEtox, including
specific usage patterns and user perspectives and
requirements when applying the USEtox model and/or
factors. All questions are assigned specific answering op-
tions. Detailed questions focus on user affiliation, how
users got aware of USEtox and how they learned to apply
model and factors, users’purpose, or field of application
for using USEtox, and finally what parts or aspects users
effectively use from the USEtox package (only applying
characterization factors, getting access to substance data,
calculating factors for new substances, etc.). Two specific
questions address the user perspective of applying USEtox
and focus on the degree of agreement regarding the per-
ceived usefulness and applicability (ease to use) of model
and factors. Whenever appropriate, an open text field was
available for additional or more detailed user feedback. We
also used the questionnaire to identify 32 users that were
interested in further discussing their perspectives. For
conducting detailed follow-up user interviews, 10 were
selected with the aim to cover different sectors and geo-
graphical regions as broadly as possible.
2.2 Detailed interviews of users and developers
Aiming at supporting the questionnaire’s outcome and
deepening our understanding of both USEtox developer
visions and user perspectives, we prepared a set of detailed
questions used for interviewing four developers as well as
4
Most potential USEtox users were invited via usetox.org. The few
potential users that accessed USEtox not directly via usetox.org, but
e.g., via a colleague’s download copy were invited via the LCA forum.
302 Int J Life Cycle Assess (2015) 20:299–310
3
USEtox developers helped to develop the user questionnaire, gave
access to the usetox.org statistics, and provided details on the consensus
building process of USEtox.
10 users from different stakeholder sectors and regions. All
interviewed users had previously completed the question-
naire. The interviews were semi-structured, allowing an
open, but focused, conversational process (Rabionet
2011). Furthermore, this method enabled us to ask in-
depth questions whenever interesting and relevant view-
points and comments emerged during the conversations.
Main focus points of the interviews were the users’acqui-
sition, application practice, and perspectives regarding
strengths and weaknesses of the USEtox model and results
with respect to its practicability. Eleven persons were
interviewed via internet phone calls, while three persons
were interviewed face-to-face.
All interviews were recorded and subsequently tran-
scribed. To link and categorize transcription fragments
with certain topics, viewpoints, or other elements in com-
mon, all transcriptions were divided into segments, which
were then coded, i.e., descriptive headers or keywords
were added to linked segments (Coffey and Atkinson
1996). This procedure gave an overview of the comments
given on different topics. To aggregate the information of
different segments per category, segments were “con-
densed”(Kvale 1996) until all information could be allo-
cated to three main categories, namely one category con-
taining user background, one containing users’perspec-
tives structured against different topics, and, finally, one
containing the developers’perspectives structured against
topics. This procedure provided an appropriate overview of
user and developer perspectives of USEtox.
2.3 Assessing developer visions and user requirements
Interviews with USEtox developers about the consensus-
building process and the visions behind developing the model
were complemented with the review of scientific publications
related to the development of USEtox. Expert review and
model comparison workshop reports (Aboussouan et al.
2004;Hauschild2006b,a; Hauschild et al. 2006; Jolliet
et al. 2006;McKoneetal.2006; Diamond et al. 2010)along
with the two framing USEtox peer-reviewed publications
(Hauschild et al. 2008; Rosenbaum et al. 2008)were
analyzed.
To identify requirements of USEtox users, we iteratively
applied and combined different requirement engineering
methods. Questionnaire and interview results were analyzed
and structured into user requests about missing features re-
garding model structure and functions and qualitative attri-
butes regarding user requests about model usability, maintain-
ability, and interoperability (Sommerville 2011). Function tree
diagrams were applied as a structural way to identify recom-
mendations based on the requested features and quality attri-
butes (Cross 2008). System context diagrams were applied to
get an overview of present contexts in which USEtox is
applied in order to help in identifying potential user interaction
improvements (Sommerville 2011). Activity diagrams were
additionally applied to visualize imagined and actual steps in
the user-technology interaction (Bhattacharjee and
Shyamasundar 2009). All diagrams were iteratively applied
and adjusted to the results from the questionnaire and
Fig. 2 Overview of applied data collection and analysis methods to compare USEtox users’practice with developers’visions and develop
recommendations for future development of USEtox
Int J Life Cycle Assess (2015) 20:299–310 303
interviews. Results of all user requirement methods were
combined to systematically compare USEtox developer vi-
sions with user perspectives and requirements, finally yielding
a set of recommendations to harmonize developer visions and
user practice in the further development of USEtox.
3 Results and discussion
3.1 Developer visions and perspectives
From development-related publications, we compiled the
original vision to develop and implement USEtox. The overall
vision behind developing the scientific consensus model
USEtox was that the data, methods, and factors of character-
izing human toxicological and ecotoxicological impacts are
harmonized and are made globally available and applicable
for LCA practitioners for a large number of chemicals. To
implement this vision, the developers aimed at establishing a
universally acceptable modeling practice and developing a
consensus-based model as joint effort of all participating
parties. This model was foreseen to (a) provide characteriza-
tion factors as strongly correlated to the factors provided by
other models as their characterization factors are to each other,
(b) produce output that falls within the output range of the
existing characterization models, (c) be parsimonious in the
sense that it contains only those elements that the comparison
of the existing characterization models identified as the most
influential, (d) provide a repository of knowledge through
evaluation against a broad set of existing models, and (e) be
endorsed by all contributors. Finally, model and resulting
factors should be more transparent and better documented
than existing tools to increase practicability and usability.
From interviews with four of the USEtox developers, we
derived more individual developer perspectives on the origi-
nal vision. The interviews revealed that not all developers
have the same perception of the overall vision in its details.
Different ideas were expressed of what is supposed to make
the model transparent for the user. For some developers,
transparency is clearly related to usability and that users with
different levels of expertise and experience are able to apply
model and factors. In contrast to that, for one developer,
transparency is related to visibility of numbers and equations
in the model to allow users to understand the modeling prin-
ciples. This originates in different opinions about the level of
user expertise. While one developer expressed that he was
satisfied with the complexity of USEtox and that he would not
encourage users without profound knowledge in environmen-
tal chemistry to apply the model, other developers want the
model to be as widely applicable to users with different levels
of expertise as possible. This would include users who only
want to use the model results without fully understanding the
model in its complexity. However, all developers agreed that
there is a limit to how easy it can be made to calculate
characterization results for which at least a basic understand-
ing of chemicals and toxicity is required. According to one
developer, guidance should ideally be available via an inter-
face that helps in the identification of required data input and
guides through the essential calculation steps. However, such
interface had not been developed since having an intuitive
user interface was not the first priority upon implementing
USEtox. The developer interviews also revealed that despite
the intent to simplify the inclusion of toxicity-related impacts
into LCA, it was unforeseen that USEtox became as widely
spread geographically and among different users across vari-
ous sectors as we can see it today with about 200 and 325
citations of the USEtox development publications at http://
scopus.com and http://scholar.google.com,respectively(the
latter representing also non-peer-reviewed literature including
books, reports, and presentations), as of June 2014. It was
further mentioned that USEtox becomes increasingly applied
and recognized also at the regulatory level, e.g., in France,
where USEtox is considered the model of choice for
ecotoxicity product labeling for the “Grenelle”legislation
(Van Hoof et al. 2011), or in the United States, where USEtox
is evaluated by the U.S. Environmental Protection Agency for
exposure-based chemical prioritization (Wambaugh et al.
2013). The developers’reflections about the use of USEtox
must be seen in the context that originally, the USEtox model
was foreseen to be primarily applied by the developers them-
selves and to provide only a list of pre-calculated characteri-
zation factors to the user community. However, in the end of
the initial USEtox development process, it was perceived
more appropriate to also allow users to calculate their own
factors, e.g., for chemicals that are currently not covered in
USEtox, thereby also providing the full model.
3.2 User application practice and perspective
Among the 551 users registered at the USEtox website, a wide
range of sectors was covered including academia (49 %) and
nonacademic research institutes (6.5 %), consultancy (18 %),
enterprises (12 %), regulatory bodies (9 %), associations
(2.4 %), private persons (1.5 %), and non-governmental orga-
nizations (NGO). The remaining 1.6 % of users did not state
their sector affiliation. Geographically, users were from Eu-
rope (57 %, where France and Denmark alone account for
almost half of all European users), North America (33 %,
mainly USA), Asia (5 %), South and Central America
(2.4 % each), and finally Australia (1.3 %) and Africa
(0.7 %). The 131 users responding to the online questionnaire
were found to cover all listed sectors (see Fig. 3a) and all
geographical regions except Africa (Europe, 67 %; North
America, 27 %; Asia, 4 %; Australia and South and Central
304 Int J Life Cycle Assess (2015) 20:299–310
America, 1 % each). All questionnaire results are summarized
in Fig. 3.
Respondents applying USEtox were predominantly con-
sultants or academic researchers, whereas the model is used to
a much lesser extent in the public sector including government
agencies, NGOs, or non-university research dominating the
“other”category (Fig. 3a). This is in line with Fig. 3d showing
that USEtox is mainly used in research including teaching
(44 %) and in management applications including life cycle
and supply chain management and corporate social responsi-
bility. Only few users apply USEtox in the context of market-
ing including public relations or regulation. The 10 detailed
user interviews revealed that users across sectors appreciate
the status of USEtox as a scientific consensus model covering
a large number of chemicals and that some researchers use the
model structure as inspiration to develop their own models.
USEtox was mainly known via colleagues or from scientific
publications, and only for less than 5 % of users via the official
website, or “other”sources including professional network,
LCA discussion forums, or conferences (Fig. 3c). In inter-
views, it was also stated that its status in the French regulation
gave inspiration for using USEtox. Most users learned to use
model and factors via the user manual (Huijbregts et al. 2010)
and the instructions directly provided in the model file
(Fig. 3b). However, several users asked for a more intuitive
user interface, supported by some interviews detailing that the
manual is difficult to understand and to apply as guide through
the modeling steps. This is consistent with the fact that almost
50 % of users do not particularly agree that “USEtox is easy to
use”(Fig. 3e) and some users even used the interviews as
opportunity to ask questions around how to apply the model.
However, the majority of users found that “USEtox is useful”
(Fig. 3e) and explained in interviews that particularly the
scientific foundation was appreciated. Almost 50 % of users
reported to only apply USEtox characterization factors and
17%toaccesschemicaldata(Fig.3f) for which the substance
data and results databases are sufficient. About 14 % of users
indicated not to directly use either model or results, but e.g.,
included USEtox as reference or list of available toxicity
models or in their teaching. Other users access USEtox char-
acterization factors via LCA software, which is especially
preferred by unexperienced users as stated in interviews, but
also by more experienced users, due to the direct use in LCA
studies. However, various users directly apply USEtox for
either calculating interim factors for fate, exposure, and/or
effects (14 %) or for calculating characterization factors
(20 %) for new chemicals not yet covered in USEtox
(Fig. 3f). These users need to understand and apply the model
Fig. 3 Distribution of answers to the questions posed in the online questionnaire to USEtox users. In questions B,D,andF, multiple choices were
allowed. *Responses to all categories but “Other use”.**Additional responses to specify further uses in category “Other use”
Int J Life Cycle Assess (2015) 20:299–310 305
itself. Interviews uncovered that some users experienced prob-
lems because USEtox results are not integrated in all LCIA
methods. This has implications in the form of inconsistent
substance coverage in the case of LCAs including chemicals
found in other models than USEtox. Along with that, it was
stated to be problematic especially for non-experts how to
correlate or compare USEtox results with results from other
LCIA models for toxicological impacts that were, e.g., used
before USEtox was available. Finally, some users indicated
via their interviews that they had problems with implementing
USEtox results into LCA software, thereby missing a way to
automatically update the software whenever they calculated
new characterization factors.
From evaluating questionnaire and user interview results,
we are able to categorize users into five actual user types with
specific characteristics based on their application field, exper-
tise, and USEtox application practice (Table 1).
LCA software developers and instructors do not necessary
apply USEtox as practitioners in LCA case studies or for
research, but they constitute important user types, since they
help in implementing USEtox results into other tools includ-
ing LCA software (LCA software developers) and/or guide
practitioners in applying model and results and might even
recommend USEtox to other users (instructors). From their
close contact to different user fields, instructors hold valuable
knowledge about user requirements, which was also a benefit
in our questionnaire and detailed interviews.
3.3 Comparison of developer visions with user requirements
The overall vision that methods and factors to characterize
human toxicological and ecotoxicological impacts in LCIA
should become globally available has been achieved within
the first years after publishing USEtox. Users apply model and
factors in several contexts, sectors, and regions, partly because
of its consensus status (see also Fig. 3). However, the vision to
be more transparent and better documented than existing tools
to increase practicability and usability has only partly been
achieved as shown from user experiences and expectations in
the previous section. As an input for potentially improving the
usability of USEtox, we therefore conducted a more detailed
analysis of user requirements. Figure 4illustrates how func-
tion tree diagrams, system context diagrams, and activity
diagrams were iteratively applied to structure usability-
related user requirements based on the data from the question-
naire and interviews with users.
In a function tree diagram (Fig. 4a), we propose possibil-
ities to improve the graphical user interface (GUI) of USEtox
towards a more intuitive and transparent application and give
examples of the level of increasing applicability, such as to
adapt the GUI until a specific user has gained a certain level of
expertise to apply model and factors without any manual. This
can be achieved via a stepwise GUI guidance system that is
accompanied with hints of where to, e.g., find and insert
relevant input data. Combining requirements of different user
Tabl e 1 Identified USEtox user types and their characteristics
User type User type characteristics
Basic user −Prefers to access/apply USEtox results via LCA
software
−Sometimes needs to calculate characterization
factors for chemicals not covered in USEtox in
LCA studies or as exercise
−Has difficulties to correlate/compare USEtox
results with results from other LCIA models
assessing toxicological impacts
−Example users: students, employees of
manufacturing companies, early stage researchers
Experienced user −Prefers to access/apply USEtox results via LCA
software
−Sometimes needs to calculate characterization
factors for chemicals not covered in USEtox in
LCA studies (scientific content is important, but
has to be pragmatic)
−Time to find characterization factors is often
limiting factor in user’s work
−Example users: experienced consultants,
employees of manufacturing companies
Researcher −Is interested in/needs access to specific features of
USEtox model and results
−Scientific purposes to apply and/or study USEtox
−Reviews and analyzes model and results in detail
(scientific content and correctness are very
important)
−May use USEtox as inspiration to develop new
models
−Example users: more or less experienced
researchers in university, other research institutes,
and consultancy companies
LCA software
developer
−Is interested in how USEtox is integrated in LCA
software and LCIA methods
−Has full understanding of LCA software and
underlying databases
−Uses/needs access to background material (raw
data, data documentation)
−Example users: developers of LCA software and
databases
Instructor −Assists (LCA) practitioners in applying USEtox
model and results
−Does not apply USEtox as practitioner, but
understands its functionality well from profoundly
studying model and results
−May recommend practitioners to apply USEtox as
function of his (instructor) own credibility in
model and results
−Has good overview of users and their application
fields of USEtox
−Example users: employees of governmental
agencies
306 Int J Life Cycle Assess (2015) 20:299–310
types (Table 1) with the contexts in which users apply USEtox
yields a specific set of interconnected subsystems illustrated in
the system context diagram (Fig. 4b). Users typically interact
manually (denoted “M”) with the front end subsystem for
inserting user input and reading model output, whereas other
subsystems like model equations describing specific fate pro-
cesses are usually of less importance for direct user access. A
detailed proposal of an improved procedure of users
interacting with different USEtox subsystems is presented in
the activity diagram (Fig. 4c). Starting with searching for a
specific chemical of interest, this diagram guides the user
through the different steps until the desired result (e.g., a set
of characterization factors, CFs) is reached, thereby passing
various subsystems. Missing data and extrapolations between
data are also included as requiring further guidance.
All diagrams were iteratively adapted until a satisfactory
level of detail was reached to transform questionnaire and
interview results into recommendations for improving
USEtox from the user perspective.
4 Recommendations
Recommendations to guide future development activities
of the USEtox consensus model with respect to user appli-
cability and functionality are designed on the one hand to
be in line with the developers’original vision to extend the
application of characterizing the toxicity of chemical emis-
sions in LCA. On the other hand, our recommendations are
designed to help facilitating the correct use and
interpretation of the USEtox model and results in different
user application contexts. Six specific recommendations
were developed:
1) Generally, the USEtox package should contain features to
allow all user types to open model and factors, perform
the calculation of intermediate and final results for imple-
mented substances, interpret all results, and—if appropri-
ate—insert new substances and/or customize landscape
and substance data. Each user type has a different level of
understanding of underlying data and methods
(see Table 1) and, hence, requires a user type-specific
level of detail in the guidance material (see Fig. 4a, b).
2) More specifically for basic users (see Table 1), a model
user interface should be provided as detailed guidance
system allowing to follow different calculation steps and
other actions step-by-step including interpretation of in-
termediate and final results, implementation of new sub-
stances, and customization of implemented substances
and landscape data (see Fig. 4c). This would help to
improve the acceptability of toxicity assessment with
USEtox among affected users. Furthermore, USEtox re-
sults should be consistently incorporated in all relevant
LCA software systems.
3) More specifically for LCA software developers and
instructors (see Table 1), additional guidance and com-
munication options should be provided by the USEtox
developers to simplify the interpretation and manual or
automatized implementation of final results (i.e., char-
acterization factors) into LCA software tools and
LCIA methods.
Fig. 4 Function tree diagram (a), system context diagram (b), and activity diagram (c) as applied to user questionnaire and interview results for
iteratively analyzing usability aspects of USEtox
Int J Life Cycle Assess (2015) 20:299–310 307
4) It should be clear and transparent how users can contrib-
ute to improving (updating implemented data upon the
availability of, e.g., improved substance data), correcting
(finding bugs in the technical functionality, errors in data
and/or equations), and further developing USEtox by, for
example, extending substance coverage and/or model
scope. Any update, however, should be in line with the
consensus status of model and factors.
5) In support of further improving and further developing
USEtox, a clear user communication and information
strategy needs to be established by the USEtox devel-
opers. More specifically, dedicated user meetings and
forums allowing for direct contact between users and
developers should be established to improve user feed-
back possibilities that can be considered in future devel-
opment steps.
6) The scope of USEtox in terms of substance, compart-
ment, exposure pathway and effect coverage, and disag-
gregation should be increased to facilitate an extended
application of model and factors in LCA studies. Howev-
er, all additional aspects should be implemented in accor-
dance with the consensus-building quality criteria de-
tailed in (Hauschild et al. 2008; Rosenbaum et al. 2008).
These recommendations have already been particularly
useful for understanding actual user needs that could partly
be considered in current update, improvement, and outreach
activities around USEtox. From generalizing USEtox-specific
recommendations, we derived the following three recommen-
dations from the user questionnaire and interview results,
which have implications for the scientific model development
process in general:
1) As part of developing scope and context of a model,
developers should familiarize themselves through differ-
ent types of dialogues with the backgrounds, levels of
detail regarding scientific knowledge and technical know-
how, and application fields of all actors they imagine as
potential users. This can be facilitated by applying Actor-
Network Theory methods. Requirement engineering
methods can then be used to define appropriate user
interfaces along with required guidance and documenta-
tion material (see (Fig. 4), thereby improving interpret-
ability and applicability aspects and model integrity and
reliability from the user perspective. This is relevant for
all types of model development, including the develop-
ment of software-based models as defined by van Vliet
(2008).
2) Depending on the desired accessibility, dissemination,
and application context of a scientific model, a clear,
transparent, and logical revision and update procedure
should be an inherent part of the model design. Users as
well as developers will benefit from this strategy as on the
one hand maintainability and testability will be increased,
while on the other hand strengthening the flexibility re-
garding different user types and application scopes. This
is mainly related to revision of software-based models
(van Vliet 2008).
3) Along with underlying scientific robustness and correct-
ness, it is recommended to integrate the technological
context of a scientific model into the design and develop-
ment phases. Aspects of reusability based on a modular
model structure, interoperability and portability between
different software and operating systems, and finally
technological interface design for incorporating parts of
a model or its results into relevant software or databases
are here equally important. This is mainly related to
software transition as defined by van Vliet (2008).
5 Conclusions and outlook
Our experiences from the detailed and complex analyses of
user expectations and experiences with USEtox and the
further development of USEtox based on these analyses
show that understanding the interactions of users with and
requirements on a scientific model and the comparison with
the developers’visions about users and model application
can guide the further development process. The variety of
user types with their differences in specific expertise and
application contexts plays a significant role in designing
model guidance material. While some of our recommenda-
tions might seem intuitive, we provide a consistent and
formal analysis of the relationships between user expecta-
tions, developer visions, and tool applicability. Thereby, we
ensure that no important relationships are ignored even
though they are not intuitive. This is in line with the ratio-
nale of using LCA as comprehensive scientific method
yielding results that might, insomecases,alsobeintuitive,
while, in other cases, revealing rather unexpected conclu-
sions (e.g., Quantis 2011). A limitation of our study is the
restricted number of surveyed and interviewed users, where
additional users with their specific requirements and prac-
tices might provide additional insight into existing applica-
bility and usability issues and constraints, expectations, and
experiences. On the other hand, the respondents offered a
reasonable coverage of the different known user types,
sectors, and geographical regions. The consensus status of
USEtox is generally much appreciated by interviewed
users, whereas some of the consensus-building criteria,
such as well-documented model and factors, are still not
met. We conclude from the results of our analysis of the
restricted set of USEtox users that usability aspects are as
important as scientific correctness to build trust among
users and to facilitate a broad and meaningful application
308 Int J Life Cycle Assess (2015) 20:299–310
of model and factors. While a more transparent communi-
cation strategy with the user community is still desirable
including a clear time plan for future updates and releases,
current improvement efforts have already lead to features
that were requested by surveyed users. These efforts in-
clude the implementation of a user forum with regular input
by the USEtox team and a frequently asked questions
(FAQ) page (part of the redesigned USEtox website), reg-
ular USEtox community of users’meetings at international
conferences, and a form and procedure to propose and
adopt improvements or updates of model and/or factors
(see http://usetox.org). The development of a USEtox user
interface wizard that will provide guidance regarding
model calculation steps and implementation/customization
of substances is in progress as this was requested by various
users. Furthermore, USEtox-based characterization factors
are implemented in several LCIA methods including IM-
PACT World+ (Bulle et al. 2012), TRACI 2.0 (Bare 2011),
CML-IA (Guinée et al. 2002), and recommended in the
ILCD handbook (European Commission 2011), whereas
ReCiPe (Goedkoop et al. 2009), LIME2 (Itsubo and Inaba
2012), and the earlier methods EDIP2003 (Hauschild and
Potting 2005) and CML2002 (Guinée et al. 2002)relyon
other models for toxicological impacts (of which CML2002
also proposes USEtox factors as a user choice). Since May
2013,USEtoxisofficiallyendorsedbytheUNEP/SETAC
Life Cycle Initiative (ILCB 2013). It remains to be seen
how the new USEtox features will contribute to further
improving the consideration of toxicity-related impacts in
LCA. Overall, scientific model design and development
processes can greatly benefit from a close and continuous
interaction between developers and users. The thorough
documentation of the survey and how it was performed in
order to document how the results were obtained will pos-
sibly inspire readers with aspirations of performing similar
surveys on other LCA-related tools.
Acknowledgments This work was financially supported by the Marie
Curie projects TOX-TRAIN (grant agreement no. 285286) and QUAN-
TOX (grant agreement no. 631910) both funded by the European Com-
mission under the Seventh Framework Programme. The authors would
like to thank all persons participating in the online survey and interviews
for their feedback and the USEtox development team for providing
website user statistics, which were treated confidentially.
References
Aboussouan L, van de Meent D, Schönnenbeck M, Hauschild M,
Delbeke K, Struijs J, Russell A, Udo de Haes H, Atherton J, van
Tilborg W, Karman C, Korenromp R, Sap G, Baukloh A, Dubreuil
A, Adams W, Heijungs R, Jolliet O, de Koning A, Chapman P,
Ligthart T, Verdonck F, van der Loos R, Eikelboom R, Kuyper J
(2004) Declaration of Apeldoorn on LCIA of non-ferrous metals.
United Nations Environment Programme, Apeldoorn
American Chemical Council (2012) ExpoDat2012: advancing exposure-
informed chemical safety assessment. Budapest, Hungary - June 2012
Bare J (2011) TRACI 2.0: the tool for the reduction and assessment of
chemical and other environmental impacts 2.0. Clean Technol
Environ 13:687–696
Bengoa X, Birkved M, Fantke P, Golsteijn L, Humbert S, Sourisseau S,
Van Zelm R, Rosenbaum R (2014) TOX-TRAIN: the user-friendly
toolbox for human and ecotoxicity assessment in LCA. Society of
Environmental Toxicology and Chemistry Europe 24th Annual
Meeting, Basel
Bhattacharjee AK, Shyamasundar RK (2009) Activity diagrams: a formal
framework to model business processes and code generation. J
Object Technol 8:189–220
Bulle C, Jolliet O, Humbert S, Rosenbaum R, Margni M (2012) IMPACT
World+: a new global regionalized life cycle impact assessment
method. Society of Environmental Toxicology and Chemistry 6th
World Congress/Europe 22nd Annual Meeting, Berlin
Carlson R, Erixon M, Pålsson A-C, Tivander J (2004) OMNIITOX
concept model supports characterisation modelling for life cycle
impact assessment. Int J Life Cycle Assess 9:289–294
Coffey AJ, Atkinson PA (1996) Making sense of qualitative data: com-
plementary research strategies. Sage Publications, Thousand Oaks
Cowan C, Mackay D, Feijtel T, van de Meent D, Di Guardo A, Davies J,
Mackay N (1995) Multi-media fate model: a vital tool for predicting
the fate of chemicals. SETAC Press, Pensacola, Florida
Cross N (2008) Engineering design methods: strategies for product
design, 4th edn. Wiley, Chichester
Diamond ML, Gandhi N, Adams WJ, Atherton J, Bhavsar SP, Bulle C,
Campbell PGC, Dubreuil A, Fairbrother A, Farley K, Green A,
Guinee J, Hauschild MZ, Huijbregts MAJ, Humbert S, Jensen KS,
Jolliet O, Margni M, McGeer JC, Peijnenburg WJGM, Rosenbaum
R, Meent D, Vijver MG (2010) The clearwater consensus: the
estimation of metal hazard in fresh water. Int J Life Cycle Assess
15:143–147
Dreyer LC, Niemann AL, Hauschild MZ (2003) Comparison of three
different LCIA methods: EDIP97, CML2001 and eco-indicator 99:
does it matter which one you choose? Int J Life Cycle Assess 8:191–200
European Commission (2010) International reference life cycle data
system (ILCD) Handbook : analysis of existing environmental
impact assessment methodologies for use in life cycle assessment,
1st Ed., Brussels
European Commission (2011) International reference life cycle data
system (ILCD) Handbook : recommendations for life cycle impact
assessment in the European context—based on existing environ-
mental impact assessment models and factors, 1st Ed., Brussels
Fenner K, Scheringer M, MacLeod M, Matthies M, McKone T, Stroebe M,
Beyer A, Bonnell M, Le Gall AC, Klasmeier J, Mackay D, van de
Meent D, Pennington DW, Scharenberg B, Suzuki N, Wania F (2005)
Comparing estimates of persistence and long-range transport potential
among multimedia models. Environ Sci Technol 39:1932–1942
Frechtling J (2010) The 2010 user friendly handbook for project evalu-
ation. National Science Foundation, Arlington
Goedkoop M, Heijungs R, Huijbregts M, De Schryver A, Struijs J, van
Zelm R (2009)ReCiPe 2008: a life cycle impact assessment method
which comprises harmonised category indicators at the midpoint
and the endpoint level; 1st Ed. Report I: Characterisation
Guinée J, Hauschild M (2005) State of the art description of characteri-
sation models for assessing human and ecotoxicological impacts in
LCA. Chalmers University of Technology, Göteborg
Guinée JB, Gorrée M, Heijungs R, Huppes G, Kleijn R, de Koning A, van
Oers L, Wegener Sleeswijk A, Suh S, Udo de Haes HA, de Bruijn H,
van Duin R, Huijbregts MAJ (2002) Handbook on life cycle assess-
ment: operational guide to the ISO standards. Kluwer Academic
Publishers, Dordrecht
Guinée JB, de Koning A, Pennington DW, Rosenbaum RK, Hauschild
MZ, Olsen S, Molander S, Bachmann TM, Pant R (2004) Bringing
Int J Life Cycle Assess (2015) 20:299–310 309
science and pragmatism together: a tiered approach for modelling
toxicological impacts in LCA. Int J Life Cycle Assess 9:320–326
Harty C (2010) Implementing innovation: designers, users and actor-
networks. Technol Anal Strateg Manag 22:297–315
Harvey F (2001) Constructing GIS: actor networks of collaboration. J
Urban Reg Inf Syst Assoc 13:29–37
Hauschild MZ (2005) Assessing environmental impacts in a life cycle
perspective. Environ Sci Technol 39:81A–88A
Hauschild M (2006a) Comparison of characterisation models for toxic
impacts in LCIA and development of consensus model, UNEP-
SETAC life cycle initiative task force 3: toxic impacts. workshop
in Montreal 4–5 November 2006. UNEP-SETAC life cycle initia-
tive, Montreal
Hauschild M (2006b) Comparison of characterisation models for toxic
impacts in LCIA, UNEP-SETAC life cycle initiative task force 3:
toxic impacts. Workshop in Paris 31 August - 2 September 2006.
UNEP-SETAC life cycle initiative, Bilthoven
Hauschild MZ, Potting J (2005) Spatial differentiation in life cycle impact
assessment-the EDIP2003 methodology. Danish Ministry of the
Environment, Copenhagen
Hauschild M, Jolliet O, Adams B, Margni M (2006) Comparison of
characterisation models for toxic impacts in LCIA, UNEP-SETAC
life cycle initiative task force 3: toxic impacts. Workshop in
Bilthoven 5–6 May 2006. UNEP-SETAC Life Cycle Initiative, Paris
Hauschild MZ, Huijbregts MAJ, Jolliet O, Macleod M, Margni MD, van
de Meent D, Rosenbaum RK, McKone TE (2008) Building a model
based on scientific consensus for life cycle impact assessment of
chemicals: the search for harmony and parsimony. Environ Sci
Technol 42:7032–7037
Hauschild MZ, Jolliet O, Huijbregts MAJ (2011) A bright future for
addressing chemical emissions in life cycle assessment. Int J Life
Cycle Assess 18:697–700
Henderson AD, Hauschild MZ, van deMeent D, Huijbregts MAJ, Larsen
HF, Margni M, McKone TE, Payet J, Rosenbaum RK, Jolliet O
(2011) USEtox fate and ecotoxicity factors for comparative assess-
ment of toxic emissions in life cycle analysis: sensitivity to key
chemical properties. Int J Life Cycle Assess 16:701–709
Huijbregts M, Hauschild M, Jolliet O, Margni M,McKone T, Rosenbaum
RK, van de Meent D (2010) USEtox
TM
user manual. Version 1.01
ILCB (2013) 23rd international life cycle initiative board meeting orga-
nized by the UNEP/SETAC life cycle initiative: minutes.
International Life Cycle Initiative Board, Glasgow
Itsubo N, Inaba A (2012) LIME2 life-cycle impact assessment method
based on endpoint modeling. Summary. Life Cycle Assessment
Society of Japan, Tokyo
Jolliet O, McKone TE (2012) Rapid exposure-based prioritization of
environmental chemicals using USEtox. Society of Toxicology
(SOT) 51st Annual Meeting and ToxExpoTM, San Francisco
Jolliet O, Rosenbaum RK, Chapman PM, McKone TE, Margni MD,
Scheringer M, Straalen NV, Wania F (2006) Establishing a frame-
work for life cycle toxicity assessment: findings of the Lausanne
review workshop. Int J Life Cycle Assess 11:209–212
Kvale S (1996) InterViews: an introduction to qualitative research
interviewing. Sage Publications, Thousand Oaks
Latour B (2007) Reassembling the social: an introduction to actor-net-
work-theory. Oxford University Press, Oxford
McKone TE, Kyle AD, Jolliet O, Olsen S, Hauschild MZ (2006) Dose–
response modeling for life cycle impact assessment: findings of the
Portland review workshop. Int J Life Cycle Assess 11:137–141
Mitchell J, Arnot JA, Jolliet O, Georgopoulos PG, Isukapalli S, Dasgupta
S, Pandian M, Wambaugh J, Egeghy P, Cohen Hubal EA, Vallero
DA (2013) Comparison of modeling approaches to prioritize
chemicals based on estimates of exposure and exposure potential.
Sci Total Environ 458–460:555–567
Molander S, Lidholm P, Schowanek D, Recasens MDM, Palmer PFI,
Christensen F, Guinée JB, Hauschild MZ, Jolliet O, Carlson R,
Pennington DW, Bachmann TM (2004) OMNIITOX—operational
life cycle impact assessment models and information tools for
practitioners. Int J Life Cycle Assess 9:282–288
Nuseibeh B, Easterbrook SM (2007) Fundamentals of requirements
engineering. Pearson Education, Harlow
Oscarson S, Hauschild M (2010) USEtox
TM
: a consensus model for
characterization of human and ecotoxic impacts in LCIA. Press
release. UNEP-SETAC Life Cycle Initiative, Seville
Pant R, Hoof G, Schowanek D, Feijtel TCJ, Koning A, Hauschild M,
Olsen SI, Pennington DW, Rosenbaum R (2004) Comparison be-
tween three different LCIA methods for aquatic ecotoxicity and a
product environmental risk assessment. Int J Life Cycle Assess 9:
295–306
Quantis (2011) Comparative full life cycle assessment of B2C cup of
espresso made using a packaging and distribution system from
Nespresso Espresso and three generic products. Lausanne,
Switzerland
Rabionet SE (2011) How I learned to design and conduct semi-structured
interviews: an ongoing and continuous journey. Qual Rep 16:563–566
Rohracher H (2003) The role of users in the social shaping of environ-
mental technologies. Innov (Abingdon) 16:177–192
Rosenbaum RK, Bachmann TM, Gold LS, Huijbregts MAJ, Jolliet O,
Juraske R, Koehler A, Larsen HF, MacLeod M, Margni MD,
McKone TE, Payet J, Schuhmacher M, van de Meent D,
Hauschild MZ (2008) USEtox—the UNEP-SETAC toxicity model:
recommended characterisation factors for human toxicity and fresh-
water ecotoxicity in life cycle impact assessment. Int J Life Cycle
Assess 13:532–546
Rosenbaum RK, Huijbregts MAJ, Henderson AD, Margni M, McKone TE,
van de Meent D, Hauschild MZ, Shaked S, Li DS, Gold LS, Jolliet O
(2011) USEtox human exposure and toxicity factors for comparative
assessment of toxic emissions in life cycle analysis: sensitivity to key
chemical properties. Int J Life Cycle Assess 16:710–727
Sismondo S (2010) An introduction to science and technology studies,
2nd edn. Wiley, Chichester
Sommerville I (2011) Software engineering, 9th edn. Addison-Wesley,
Boston
Takhteyev Y (2009) Networks of practice as heterogeneous actor-net-
works. Inf Comm Soc 12:566–583
Udo de Haes HA (1996) Towards a methodology for life cycle impact
assessment. Society of Environmental Toxicology and Chemistry
Europe, Brussels
Udo de Haes HA, Jolliet O, Finnveden G, Hauschild MZ, Krewitt W,
Müller-Wenk R (1999a) Best available practice regarding impact
categories and category indicators in life cycle impact assessment.
Int J Life Cycle Assess 4:66–74
Udo de Haes HA, Jolliet O, Finnveden G, Hauschild MZ, Krewitt W,
Müller-Wenk R (1999b) Best available practice regarding impact
categories and category indicators in life cycle impact assessment.
Int J Life Cycle Assess 4:167–174
Udo de Haes HA, Finnveden G, Goedkoop M, Hauschild MZ, Hertwich
E, Hofstetter P, Jolliet O, Klöpffer W, Krewitt W, Lindeijer E,
Müller-Wenk R, Olsen S, Pennington DW, Potting J, Steen B
(2002) Life-cycle impact assessment: striving towards best practice.
SETAC Press, Pensacola, Florida
Van Hoof G, Schowanek D, Franceschini H, Muñoz I (2011) Ecotoxicity
impact assessment of laundry products: a comparison of USEtox
and critical dilution volume approaches. Int J Life Cycle Assess 16:
803–818
van Vliet H (2008) Software engineering: principles and practice, 3rd
edn. Wiley, New York
Wambaugh JF, Setzer RW, Reif DM, Gangwal S, Mitchell-Blackwood J,
Arnot JA, Jolliet O, Frame A, Rabinowitz J, Knudsen TB, Judson
RS, Egeghy P, Vallero D, Cohen Hubal EA (2013) High-throughput
models for exposure-based chemical prioritization in the ExpoCast
project. Environ Sci Technol 47:8479–8488
310 Int J Life Cycle Assess (2015) 20:299–310
