Access and use of the UK e-Infrastructure: OMII-UK and NGS case studies

Full text

Access and Use of the UK e-Infrastructure
OMII-UK and NGS Case Studies
Dimitrina Spencer
Matteo Turilli
Grace de la Flor
Marina Jirotka
Sharon Lloyd
Ralph Schroeder
A report from the „Embedding e-Science Applications:
Designing and Managing for Usability‟ project
(EPSRC Grant No: EP/D 049733/1)

Contents
Contents ...................................................................................................................... 2
Introduction ................................................................................................................. 3
Methodology ................................................................................................................ 8
User Typologies ........................................................................................................... 9
OMII-UK .................................................................................................................. 11
Working with the End-users .................................................................................. 12
Working with end-user “communities” .................................................................. 19
Qualities of the developers ................................................................................. 20
Resources and Institutional Pressures ................................................................. 21
Communication and the Role of Language......................................................... 22
Documentation and Support .............................................................................. 23
Working with the Developers ................................................................................ 26
Bringing the users together ..................................................................................... 28
The National Grid Service (NGS) .............................................................................. 29
User Engagement ................................................................................................... 33
User Experiences with Certificates ......................................................................... 35
User Experiences with Middleware ........................................................................ 37
Partnership Joining Experience .............................................................................. 39
Service Deployment Challenges ............................................................................. 42
Conclusions and Recommendations ........................................................................... 44
References ................................................................................................................. 50

Introduction
There are numerous e-Infrastructure providers in the UK that offer a wide variety of
services to diverse user environments. The National Grid Service (NGS), OMII-UK,
Janet, Edina, High End Computing Terascale Resource (HECToR) and HPCx are all
examples of founded organisations that are contributing to the development and the
deployment of a UK national e-Infrastructure. An important role in such an effort is
played also by data centres, visualisation and text mining networks, digital curation
centres, HPC resources and campus grids (Geddes et al. 2007).
The term “e-Infrastructure” has been coined to describe the next generation of
research environment across Europe. In a briefing paper of May 2006,
1
JISC describes
an e-Infrastructure as the “technology and organisations” supporting “shared access to
large data collections, advanced ICT tools for data analysis, large-scale computing
resources and high-performance visualisation”, involving the integration of “operations
centres, service registries, single sign-on, certificate authorities, training and help-desk
services”. In a recent report Hardisty et al. (2009: 3-4) pointed out that an e-
Infrastructure should be:
 A secure way of working for projects using resources;
 Of common use to some extent;
 Providing a broad range of opportunities for as many users as possible;
 Able to tell you what is available and what is appropriate for your use;
 Mostly hidden from users but not a brick wall barrier for expert users;
 Usable!
The definition offered by JISC and the list of desiderata given by Hardisty clearly show
how an e-Infrastructure is a complex socio-technical system. In this system, dedicated
middleware, authentication, authorisation and accounting infrastructures, and user
applications are deeply embedded into human-driven organisations that offer service
development and deployment, user engagement, training and support.
The balance between functionalities and usability is one of the core difficulties
for the implementation and deployment of an e-Infrastructure. It comes then as no
surprise that the lack of usability has been one of the key criticisms made to the UK e-
Infrastructures (e.g. Rixon 2003, Beckles et al. 2004, Beckles 2005, Chin and Coveney
2004). In order to address this problem, this report collects the results of field-work
1
http://www.jisc.ac.uk/publications/documents/pub_einfrastructurebp.aspx

conducted within OMII-UK
2
and NGS
3
, two leading UK e-Infrastructure providers
and UK e-Science flagship projects. The analysis of the data collected has been used to
support a set of recommendations on how to improve the accessibility and usability of
these two services providers.
Since their inception (OMII-UK started in 2004 and NGS in 2003) both
providers influenced the practice and development of e-Research in UK. In the course
of their growth as national services, they have obtained three subsequent funding
rounds and have evolved significantly. Both providers have adapted to the evolving
requirements and instruments of e-Science and have also capitalised on the
accumulated experience.
Both OMII-UK and NGS have enjoyed numerous opportunities arising from
the growth rate of e-Research in UK and the consequent development of a user
community in need for a national e-Infrastructure. Nonetheless, at times, their
evolution has been constrained by a number of structural factors. They had to
reconcile their project-oriented funding policy with the need to operate in quality of
national services, they had to expand on the territory alongside the existing resources
managed by Higher Education Institutions (HEI) and they had to balance their service
provision with the needs of their users.
The goal of OMII-UK has been to enable e-Research through the use of open
source software and to provide end-users, research groups and institutions with
support and dedicated software packages. OMII-UK funds the development of new
software liaising with national and international organisations and e-Research groups.
OMII-UK consists of three teams based respectively in the University of
Southampton, the University of Manchester and the University of Edinburgh and has
many national and international collaborators.
The goal of the NGS is to provide and facilitate the robust, reliable and trusted
access to computation and data resources through a national grid system. The NGS
grid resources are made available via the Internet by means of open standards so to
support distributed collaborative research across all UK. NGS is led by the Science and
Technology Facilities Council (STFC) and coordinated by the Rutherford Appleton
Laboratory (RAL) in collaboration with the University of Manchester, the University of
Oxford and the White Rose Grid at Leeds. NGS resources are provided by these four
founding members alongside HPCx, HECTOR and a growing number of partners and
2
http://www.omii.ac.uk/
3
http://www.ngs.ac.uk/

affiliate sites. NGS can count also on a wide network of national and international
collaborators.
Both OMII-UK and NGS have begun their third development phase in 2009.
Their new goals and the way they will work towards achieving them will determine the
impact they will have on the future of e-Research in UK. Both service providers are
now focusing on the development and the delivery of high quality, production level
services and applications to their current and potential users. In this context, what they
do to ensure the usability of their services and software is critical for their success.
While in the past years the key challenge was to build and provide the critical
components of a national e-Infrastructure, today the task is to make such an e-
Infrastructure stable, well-designed, efficient, supported, scalable, widely and easily
usable.
In order to provide viable recommendations about how to improve access and
usability, this report offers an insight into how OMII-UK and NGS have engaged with
their current and potential users and how they are striving to:
 Ensure wider uptake;
 Deliver production level services and applications;
 Manage their procedures in order to guarantee a high standard of quality of
service and support.
Specifically, we examine how NGS and OMII-UK are interacting with end-users,
developers, research groups and institutions focusing on four stages of their service
delivery process:
 Initial user engagement;
 Requirements elicitation and implementation;
 Service evaluation; and
 Support.
Our findings underline the key role played by the interaction between providers and
users at each stage of service development and delivery. We consider the impact that
particular forms and methods of user engagement have on the usability of OMII-UK
software and NGS services. Moreover, we discuss how various external and internal
factors may be limiting or enhancing the mutual relationship between providers and
users. Overall, our analysis, findings and recommendations confirm how crucial it is to

pay attention to and provide further understanding about the human component of the
national UK e-Infrastructure (Lee et al. 2006).
The users of the UK e-Infrastructure have emerged from our field-work as
active agents in initiating the use and development of the e-Infrastructure but it
appeared also clear that it is essential for the service providers to find efficient ways to
implement their strategies. In this context, we argue that the attitude with which the
provider/user relationship is built and sustained is critical. For this reason, we suggest
that infrastructures should be developed “with” rather than “for” the users.
This report draws on the existing research experience about the nature and
purposes of e-Infrastructures and the strategies for wider uptake of e-Research. We
explore the link between accessibility, usability and e-Infrastructure drawing
predominantly on Beckels (2005) and Ure et al. (2008). We focus on understanding
how technical and social factors are interlinked in the development, deployment and
use of the UK e-Infrastructure, specifically for what concerns OMII-UK and NGS.
Our research is part of a series of projects investigating the social factors
involved in the UK e-Infrastructure development, particularly in relation to the barriers
and opportunities to adoption. Some of the key initiatives in this area include Oxford
e-Social Science (OeSS, Grant No. RES-149-25-1022), SUPER (Newhouse et al. 2007),
AVROSS (Barjak et al. 2008), e-Infrastructure Use Cases and Service Usage Models
4
(eIUS), Enabling Uptake of e-Infrastructure Services (eUptake) (see Voss et al. 2008)
and Engaging Research with e-Infrastructure
5
(ENGAGE).
Most of these projects aim at translating the user‟s experiences to providers.
For example, the goal of e-Uptake project is to “provide service providers and
technology developers with a sound grasp of problems as perceived by users”.
6
While
we agree that this aim is of great significance and fully support it, we suggest that
providers experience have remained under-researched. A better understanding of the
experience of service providers and technology developers is crucial to improve the
accessibility and usability of e-Science services and software development. Both
accessibility and usability are a two-way complex relationship that cannot be reduced to
one-way communication from users to providers and developers.
For this reason, rather than focusing on either users or providers, in this report
we explore the complexity and richness of their interrelationship. In doing so, we
4
http://www.eius.ac.uk/
5
http://engage.ac.uk/
6
http://www.engage.ac.uk/e-uptake

uncover the critical interweaving of different levels of context: the individual, local and
national as well as the disciplinary and interdisciplinary forms of collaborations. All
these contexts are characterised by different needs and modes of use that directly affect
the quality of the UK e-Infrastructure provision. These heterogeneous set of needs and
modes of use, their relational nature and origin have then to be taken into account
when reasoning about how to improve the accessibility and usability of such
infrastructure.
In so doing, we follow the existing research on “collaboratories”
7
and remote
use of scientific instruments that has brought a number of important insights as to
their socio-technical nature (Finholt 2002, Kling et al 2003). Lee et al. (2006) examine
the importance of human infrastructure in terms of „people, organisations, networks
and arrangements that constitute our site as a collective entity‟ and rather than studying
it simply in terms of structures of functions, they employ ethnographic methods in
order to see “the infrastructure in use‟‟. Although the authors magnify the social aspect
by focusing on the “human infrastructure,” they adhere, as we do, to the understanding
that the social and technical aspects of infrastructure are interwoven.
Following this approach, we describe some of the important social practices
and socio-political conditions in the emergence of an e-Infrastructure. The focus on
social relations allows, for example, the exploration of how both users and providers
rely on and manage the human infrastructure in order to compensate for the
inadequacies of the technical one. What the technical parameters themselves cannot
allow, the human relationships, and particularly the face-to-face interactions often can.
Technology works well in terms of accessibility and usability only when it is properly
integrated into the human infrastructure so to create an efficient and coordinated
socio-technical system.
Our report shows the key importance of many different types of user
engagement in promoting and enhancing accessibility and usability of the UK e-
Infrastructure. The problem is that user engagement and, particularly ongoing end-user
engagement at all stages of the e-Infrastructure development, continues to be a
challenge for service providers. In analysing the key ways in which OMII-UK and
NGS engage and interact with their users, we uncover some of their main challenges
deriving recommendations for how accessibility and usability could be improved. The
7
The term collaboratory was introduced by Wulf (1993) as „a laboratory without walls‟ allowing
collaboration across geographic distances. See also Science of Collaboratories (2003) for a recent definition.

success of this report will be measured by the debate it will promote inside OMII-UK
and NGS and by its impact on widening the sustainable uptake of e-Research.
The report is divided into five sections. The first offers a description of the
methodology adopted in our field-work and data analysis. The second contains an
overview of the user typologies of the UK e-Infrastructure with particular attention to
those of OMII-UK and NGS. This is not a list of users but a categorisation that has
been developed in order to code the data collected in our qualitative research. The
third section of the report is dedicated to OMII-UK and the fourth to NGS. These
sections contain the analysis of the data we have collected about the interaction users-
providers. Accessibility and usability of the services provided are at the centre of the
analysis. Conclusions and recommendations close the report.
Methodology
Our research is based on qualitative data gathering and analysis. This qualitative
approach complements the several studies that have been mentioned above that have
produced extensive quantitative data (and sometimes also qualitative findings) about
user engagement. The data for this report were collected through ethnographic studies
of e-Science projects including interviews both with providers and users. The
ethnographic observation focussed on the work practices and organisational processes
of e-Infrastructure provision and the relationships between users and providers. The
data collection took place between July 2008 and July 2009 and, thus, captures only a
limited period of time. At the same time, some of the data also point to the important
role of the early history of OMII (then OMII-UK) and NGS as well as to their future
directions.
Some of the limitations of the study include the intermittent access to the project
teams, which are distributed and spread across the UK. Thus, the method of multi-
sited ethnography (cf. Marcus 1995) was employed and the locations of study were
chosen in relation to specific events on the projects within the constraints of expense
and ease of travel. Some mundane interactions were observed in ongoing project
meetings via the Access Grid audio-video teleconference system, on the phone or in
the project offices. Specific interactions with research participants included:
1. Face-to-face or telephone interviews and observations;
2. Teleconference observations;

3. Observation and data gathering at user forums, annual workshops and project
staff meetings;
4. Website presence;
5. Email communications.
When possible, data from the projects and users has been anonymised and permission
for making this report public has been sought from the research participants.
The interviews were unstructured (particularly at the beginning of the research)
or semi-structured (when certain issues have been elicited for examination). The
verification was conducted through triangulation of data and the in-depth
understanding of the mechanisms impacting on usability. More than 30 semi-structured
interviews have been conducted but the amount of in-depth discussions during
participant observation exceeds this number significantly.
In the course of research, the circle of interviewees has expanded. Comparing
different types of users and projects has led to important insights on how different
levels of context and actors interrelate. In this way, we have been able to describe the
complexity of the mechanisms affecting accessibility and usability and the ways in
which they remain interdependent.
User Typologies
The ongoing evolution of certain research methods (e.g. the rise of computational
sciences or the growth of large, interdisciplinary research projects) has created new
demands on scholarly practices and, as a consequence, the raise of new requirements
for some type of academic research. The creation of a national (and sometimes
international) e-Infrastructure has been the answer to the needs for computational
power, data resources and collaboration. The users of such infrastructure are
individuals and different types of institutions involved in research projects that need to
harness the opportunities of (inter)nationally distributed ITC resources. Typically, their
goal is to achieve a more efficient research practice, to gain access to the resources
necessary for computationally-based science and the creation and use of large
databases but also a promotion of the institutional reputation and status, the fulfilment
of a political agenda or an increase of the institutional research output.
As pointed out by Lee et al (2006) there are four important processes related to
the emergence of the new forms of scientific collaborations:
1. The rise of „big science‟ since WWII (Galison 1992 cited in Lee et al. ibid.);

2. The rise of interdisciplinarity;
8
3. The political and economic practices defining prestigious research; and
4. The spread of advanced technologies and the virtual sites in research
practices.
While these trends have been examined well in the existing literature, the research on
the social practices and the coordination of collaborations is more recent (see e.g.
Jirotka et al 2006, Olson et al. 2006 and Ribes and Bowker 2006, Finholt et al. 2006,
Bechki 2006). In the following sections, we explore predominantly how different forms
of users‟ collaborations may be related to the development and adoption of software
and grid-enabled services.
In the course of our observations we have outlined two main classes of users:
1. Individuals:
a. Researchers;
b. Developers;
c. IT professionals.
Individual users can be member of a research group or institution but they
interact with service providers as a single user or is regarded by the providers as
such. In the context of e-science, it is often assumed that the end-users are
researchers but it is important to stress that developers and IT professional
should not be overlooked. This is particular relevant when considering that
many e-science projects involve software development.
2. Institutions:
a. Higher Education Institutions;
b. Research groups;
c. Service providers;
d. Private companies.
These are institutions that interact as a whole with OMII-UK and/or NGS or
are treated as such by them. Specific services are requested or tailored by or for
the whole institution. The provided solutions are successful only when the
institution as a whole is satisfied.
8
We have also drawn on research on interdisciplinarity and its effects, highlighting the importance of
organisations, context of knowledge production, funding and multiple stakeholders (Klein 2004, Palmer
2001, Cummings and Kiesler 2003, Jeffrey 2003, Gibbons et al. 1994).

This user taxonomy is not only consistent with our observations but also with the
current literature. For example, Newhouse et al. (2007) describes users of an e-
Infrastructure as “anyone conducting research, design, analysis or diagnosis across all
disciplines covered by all the Research Councils”, not only individuals but also
“institutions, such as universities and publicly funded research institutes”.
NGS and OMII-UK classify as service and technology providers, the former
offering grid services while the latter directly or indirectly supporting the development
of grid applications. Software development support and collaboration are part of the
services offered by OMII-UK to its user community. The methodologies adopted by
OMII-UK determine the way in which this service is delivered and managed. In the
following paragraph, the OMII-UK approach to software development is analysed in
order to understand how accessible and usable such a service is. The same is done for
NGS and the way in which support is given to institutions in order to install, configure,
integrate and deploy grid middleware and the NGS software stack.
OMII-UK
OMII was established in 2004 with funding from the EPSRC and until 2006, when its
first round of funding expired, it focused mainly on developing middleware software
for grid computing. In 2006, OMII-UK was formed and some internal differences
about the goal of the project emerged. Part of the staff thought that the focus of
OMII-UK should have been on better testing and Quality Control (QA) while others
argued for a more rapid development process and release cycle. The latter view was
taken on board with the consequence of shortening the software release cycle from 4-6
months to 2-3 months. This change in the development practice alongside an increased
attention to problems of interoperability was thought to better suit the requirements of
the OMII-UK users.
With the change of management in mid-2007, the new managers of OMII-UK
changed direction once again, focusing more on consolidating the existing middleware
and on the provision of solution-based software. In this process, they aimed at
establishing OMII-UK as consultants who provide advice on software evaluation and
integration rather than just attend to software product development. This has involved
three main areas in which OMII-UK have begun to deliver:
1. The Campus Grid Toolkit;
2. The Data management Solutions;

3. The Bioinformatics Workbench.
These changes of direction through the course of the development of OMII – called
eventually OMII-UK – as an organisation has impacted on the way OMII-UK
practices user engagement. While in its early days, OMII-UK was more of an internally
focused organisation, gradually, it has become more externally oriented beginning to
engage more with its users in order to understand their needs and requirements.
Furthermore, while in the past OMII-UK has worked predominantly with developers
or service providers, more recently, and especially in the process of running the
ENGAGE project, OMII-UK has begun to work directly with the end-users in the
development of specific applications and middleware.
The work done by OMII-UK on the ENGAGE project can be seen as a form
of applied collaborative research, where the dialogues with users about their needs
have led to projects where developers, users and OMII-UK work closely together with
the goal to enable e-Research. An interviewee at OMII-UK has suggested that: “our
more successful programs are working directly with the users” while another one
underlined that “if we do any development, users need to be involved. In the past we
worked more with service providers, we know them, we see them at conferences, they
provide resources, but the more successful programs are working with the users
directly”.
This increased awareness about the need to engage with the users might help a
shift towards a more user-oriented method for service development and provision in
the UK e-Infrastructure. As the OMII-UK experience shows, this shift could lead to
more successful projects and, in the long run, to a more successful e-Science national
program.
The next section provides examples of how OMII-UK has worked with end-
users such as researchers and developers and with institutions such as service providers
and research groups. These examples span across different stages of the software
development process: initial engagement, requirements gathering, implementation,
evaluation and support.
Working with the End-users
Many of the OMII-UK end-users are researchers within the e-Research community
and conducting experiments or projects that require large amount of data or
computational resources. Computational biology and chemistry, scientific visualisation,

clinical analysis, astronomy but also arts and humanities are all examples of
communities conducting e-Research while using (among others) software and support
offered by OMII-UK.
Initial Engagement: Communication and Visibility
From the OMII-UK experience with initial engagement, the first encounter with the
users and the first messages the users receive about OMII-UK may play an important
role in how their relationship develops further through building expectations and trust.
Thus, OMII-UK has taken seriously their strategies for visibility. Most often, OMII-
UK staff members makes initial contact with users at workshops or conferences or
through word of mouth but they attribute the greatest value to face-to-face meetings.
This is how one of the interviewees described the process: “attracting interest and
starting a dialogue, then establishing communication and getting the initial contacts.
Conferences and face-to-face meetings are the most efficient tool and the website
content and brochures but until they result in a face-to-face [discussion] you have not
gone very far”.
Some of the key ways in which OMII-UK provides visibility to its work are:
 Giving presentations at conferences;
 Organising a stall at conferences which acts as initial user engagement forum;
 Organising training workshops and taking part in summer schools;
 Developing and maintaining an interactive website;
 Publishing a newsletter to which researchers contribute or simply receive and
hear about what other researchers‟ experiences with OMII-UK;
 Getting in touch with different researchers throughout the world through
targeted internet search or hearing from others about them;
 Documentation of software (see the section on support further below).
 Principal Investigator (PI) networks.
 The ENGAGE Project.
The contacts that the OMII-UK‟s PIs make with other PIs are seen as extremely
important to initial user engagement. However, in the experience of OMII-UK, these
contacts lead to a successful engagement only when they are rapidly passed over to
other, less busy, project members on both sides. In so doing, the necessary time and
relevant competences can be dedicated to establish ongoing communication channels.
For example, if the PI on the users‟ side is too busy, the contact may not develop into

a partnership at all: “We want to involve them upfront and keep them involved
upfront throughout the process and we have researchers who are not PIs but know
what is important. The PIs are often away and they do not have the bandwidth to work
with us. They have to introduce us to their team members who can work with us on a
regular basis”. Users that are more directly involved in the research process and know
the needs of their project more closely seem to have made better start for collaboration
according to OMII-UK. At the same time, the role of PIs is essential as the power to
authorize the initial steps lies with them. In this sense, securing both the support of the
PI and the ongoing cooperation of end-users is an important initial step for engaging
on the right footing.
In this context, OMII-UK have realized that the public image of the project is
also very important, even more so, because it is the initial source that builds
expectations among the users and expectations may determine the possibility for
engagement. OMII-UK has made an active effort to deliver a clear message about their
mission and what they offer. As one of the interviewees put it:
“False impressions are still put about what OMII does. In 2007 we changed what
we do and some still think we do what we used to do. The PIs put across what we
do now but there is inertia in the information that is around. When OMII-UK
stated in 2006 people realised we changed but in 2007 they still thought we worked
as in 2004 so our PIs went around to clarify what we are about now: OMII is about
software solutions for e-Research. Before it was: a large piece of integrated software
stack that did not perform a useful function or rather it did but there were pieces of
function that performed it better. Now, we collect components either individually
or together and other systems and connect researchers to computational power and
storage”.
In order to give a unified message to their current and potential users, OMII-UK has
paid attention not only to the quality of written messages they give about their work
but also to how the OMII-UK staff members perceive their mission. Specific
organisational effort is devoted to establish a common vision about what OMII-UK
does: “The role of management internally is to make it clear what our message should
be. If someone phones up (who does not go to conferences), it is important we do not
contradict what we say in public. Internally, I have to do an awful lot of internal
dissemination”.

OMII-UK management makes sure on a regular basis that all staff is aware and
able to communicate the current affairs of the project and what OMII-UK or its
services are about. This is achieved through regular staff meetings where information
about each team and individual team members is circulated so to renegotiate the
overall vision when necessary. In the experience of the staff we interviewed, clarity
about the vision of OMII-UK seems to support the establishment of realistic user
expectations and it is a prerequisite for working towards usability.
Apart from expectations, the initial stage of user engagement is also about the
building of trust with the end-user: “The environment we work in – we need a bit
more trust. We want people to use OMII software for their research. We would like to
have trust and reputation – e.g. people who do not know us – to find out easy about
what we do and to overcome any barriers for access and then use our consultancy to
solve their problems”.
Because e-Research is still novel, often, the initial user engagement requires
informing the users about the possibilities of e-Research in general. For example,
often, OMII-UK staff are asked to explain what “cloud” means in computing. For this
reason, OMII-UK have found the idea of creating a dictionary of e-Research terms
very useful so that they could refer the users to it or use it in conversations with the
users. However, it is questionable whether such a dictionary will be able to truly
standardise the meaning of concepts which denote fast changing phenomena such as
those pertaining to e-Research. Perhaps, the main issue to address here is how to
dialogue with users in a dynamic environment lacking strict definitions, and
particularly, how to dialogue with users who do not have much computing knowledge.
In this context of novelty of e-Research and wide user base, some OMII-UK
staff has sometimes struggled to find the best way to articulate what e-Research is and
how they can enable it. Some of the staff members have expressed the need for wider
support by the research councils of the e-Research vision they promote. For example,
one of the interviewees suggested: “We should ask the Research councils to publicise
what we do because scientists trust what research councils have to say. If I am a
scientist and have OMII-UK call me – I do not know who they are, but if research
councils say „work with OMII-UK if you are in this area‟ – they will trust”.
In order to build trust and manage expectations, OMII-UK staff members not
only put an effort to communicate clearly what OMII-UK offers but also aim to
understand the needs of their potential users. In OMII-UK‟s experience, while the

means of communication here are important, it is even more important to be realistic
about the potential outputs from the very beginning. In understanding the initial user
requests, OMII-UK may work directly with the end users or through mediators. For
example in some cases they have appointed a “technical reviewer”, that is, someone
“who does not do development but acts as advocate of users sitting on the project”.
This approach has been particularly useful for OMII-UK when collaborating with
UCL. The technical reviewer, being locally based at UCL, could talk to the end-users to
determine the initial requirements directly from the end-users. He produced the initial
set of requirements and as they began to evolve and needed refinement, OMII-UK
staff got directly involved with the end-users: “being close… this is how you elicit what
they require and understand the key points. The more traditional way is to get a list of
requirements but understanding the domain is how you can better meet the
requirements… with this understanding you can do a lot better”. In other cases, the
role of the initial mediator may be played by an OMII-UK staff member who has the
personal talent to communicate well with new end-users, gain an understanding of
their needs and report them back to the developers.
The initial requests of the users vary in terms of specificity and technical
sophistication. Spending time to properly understand users‟ requirements and to reflect
on what is possible to achieve within the development project is an effective
preparatory effort for the following stage of requirements elicitation.
User Engagement during Requirements Gathering and Development
The practice of user requirements gathering has varied during the different stages of
OMII-UK and within the different projects depending on the users involved. Today,
OMII-UK see requirements gathering as ideally involving the users as active
participants but, in practice, sometimes users continue to be less involved. In this
section, we present some examples of how OMII-UK involves users in requirements
gathering and development. In our analysis, we point out what might be some of the
underlying challenges.
As one of the OMII-UK staff members described it:
“We will normally write a project plan stating what the requirements are; what work
will be done; the time scale and who is involved; who does what bits so that there is
ownership. We may have ownership of some parts and the users may have
ownership of other parts of the work. For example, split normally happens on small
scale projects: we or those who are technical take responsibility of development and

users are responsible for evaluation. Because we have expressed the requirements in
the project plan we need to know the criteria to meet. As part of the ongoing
communication process, you will have the ability to assess the requirements and see
if they are still good. At the beginning there may be many requirements; as you go
some become of lesser a priority. On a recent project, we thought that integration
of software was a high priority but as the project evolved we found that actually this
was not a high priority. A higher priority was to focus on a better configuration of
one of the pieces of software. And the way we found this is that we had a
misunderstanding of how the software will be installed and deployed. We thought it
will be installed by system administrators but we discovered that those installing
were researchers, not system administrators and this is why the priorities changed.
We changed the remaining work plan to respond to this and when we come to
evaluation we will give a different ordering of priorities and we would have satisfied
them”.
This quote underlines the importance of an iterative approach to software
development based on ongoing communication and requirements assessment between
developers and users. In this context, a clear definition and alignment of the interface
between OMII and users becomes a precondition for the success of the development
effort led by OMII.
From the interviews we had with OMII-UK staff, it seems that they see the central
ingredients for a successful requirements elicitation as:
 User involvement (including the qualities of the user);
 Communication (including language);
 The qualities of the developer (or whoever is communicating with the users
from OMII-UK);
 The role of PIs;
 The available resources;
 (Most importantly) the method of requirements gathering
Where possible, OMII-UK turns towards agile and user-centred development although
some development continues to take place with limited interactions with or
understanding of the users. In the following sections, keeping in mind that the
development effort of OMII-UK is a core service offered to the user community, we
explore briefly some views of OMII-UK staff members about engaging the users in

their development practice as well as some of the constraints they meet in achieving
their goals.
The importance of users’ “availability”
The following interview excerpt illustrates how some OMII-UK staff members
understand the practical benefits of user-cantered design. This approach to software
development helps to produce not only better results for the current users but also to
guarantee the future ones:
“Until you get an actual end user involved nothing goes forward, an active end-user
focuses the effort and guarantees better result. If you work with a particular user in
mind and make him happy, it is better than doing a generic installation that no one
finds it works well for them. Now, we may go to other groups to UCL and because
research is distributed if we do a good job for them, may be their colleagues in other
universities will come to talk to us”.
For the success of this type of user engagement, the qualities of the actual users were
highlighted as particularly important because, only an active and available user could
allow for a user-centred design or agile development to take place. Here, “available”
designates a user who can make the time to give ongoing feedback and to take part in
meetings, being responsive with email, getting phone calls, and so on. It is also
important that the users involved in the development process have the power to take
and implement decisions or can ensure access to such power. Finally, these users have
to have the interest and the motivation to be an equal partner in the development
process. A member of the OMII-UK staff stressed this point saying that:
“Sometimes we run into technical problems and then we have to manage user
expectations so we try to do agile development delivering small increments regularly
and getting evaluation of those users. They have a commitment and they have to see
what we have delivered. Where this process breaks down [is when] the end-user
may be not interested or be not senior enough [cannot make decisions] and even if
the developing team is delivering but the end user says: «I am busy and cannot look
at it» then the agile process cannot work. You need the feedback, it is a two way
thing. The big bang approach is: you start the project, never talk to each other,
software developers talk to each other and deliver but developers do not know what
the users need and the software will not be used. If you deal with an end user: «Send
me something and I will evaluate it», you have a chance to be successful”.

User-centred design and agile development are a time-consuming two-way process and
neither will work if the users do not accept to play their important part in the process.
The challenges of learning what it means and how to engage closely face both
developers and users. As a consequence, the responsibility in this process is to be
shared between them.
Working with end-user “communities”
Gathering requirements from small-scale projects or individual users differs from
working with end-users who represent “communities of users”, meaning disciplinary or
sub-disciplinary branches. Working with the latter is often about “creating portals [for
example] for chemists to allow them to use different protein modelling software or
providing better translation of different data bases for humanities researchers working
with commentary-based databases”. The requirements gathered are more generic and
the methods used by OMII-UK for their gathering are different. For example, in the
case of the European transnational research initiative called “Systems Biology of
Microorganisms” (SysMO), an OMII-UK developer explained that:
“[Within] SYSMO we are looking at the provision of overall infrastructure and
using web technologies for systems biologist. This is a larger scale project and we
have a more general set of requirements. The requirement there is to have technical
knowledge; to have domain knowledge - to understand the information
infrastructure of that community and to create a common language between
ourselves and the community”.
Designing for a “community” poses a specific usability challenge: how to balance
generic versus specific usability requirements: “it is harder to understand how to
balance requirements”. OMII-UK has recently made a shift in their way of working on
this kind of projects:
“the way we deal with usability has changed while three years ago we would have
tried generic improvements – to create pieces of software useful for many groups –
now, we need to do targeted development and transfer one at a time to another
community, that means, specific usability improvements from community to
community, things that meet requirements”.
This new approach is based on a modularised, incremental development approach that
has direct impact on the way usability is achieved across similar communities:

“the process is to forward the requests somehow and to ask the original requester if
other people have similar requests and for us to identify whether different
communities would make the same request - they may have made similar requests;
[for example] the way different communities are using TAVERNA workflow are
different from how the original community used it; they wanted longer workflow -
so you had to run them as a server not on the workstation and this came up from
three different communities; this translated to all communities as it is a common
requirement; then this work was clearly of a higher priority [for us] and was
scheduled into development; […] the portal will be redone for everyone‟s need to
improve the efficiency; [so we need] to make it easier to create new portals and so
we try and make improvements in portal creation; it would be better rather than
make a portal that does not work for anyone; the usability will be poor for everyone
on a generic one”.
This quote clarifies that software development process needs to be constantly adapted
to new requirements coming for a growing user base. It is critical in this process of
adaptation to balance generic and specific usability requirements so to achieve
flexibility without impairing efficiency.
Qualities of the developers
As OMII-UK have begun to engage more closely with end-users, they have realized
that the key to understanding users and delivering usable products might often be in
the qualities of the developer who works directly with the users. Here is a familiar
problem: “when we first sent a developer - they [the users] did not understand him […]
software developers do not give introductions [but users] need to know what the
developers will talk about”.
OMII-UK has found that apart from the technical expertise, developers should
also possess good facilitation and communication skills and an ability to understand the
domain of the researchers and translate back and forth between his team and the users.
Some have argued that often, the developer who talks to the users does not even have
to be the best of developers as long as they can understand and make themselves
understood to the users. This is how the ideal developer was described in some of the
interviews:
“They can understand what the kind of infrastructure requirements are. They are a
bit like consultants. They do not know the subject area but can talk to the people in
the subject area. It is a certain type of person: they have to be systems analyst or

applications consultant and they have to be good in talking to people and doing
iteration; listening to someone and trying to summarize and express and get a
confirmation that this is what is meant”.
It seems though that not all developers possess such talents and those who can do this
job well may not sometimes be available to work with users as their time may be
allocated on a different task. All OMII-UK staff members interviewed agree that
without such staff effort at all stages of a project, and especially at its beginning,
usability issues might be at stake. These considerations clearly point out the critical role
played by the training of developers and software engineers.
Our observations support the need for both undergraduate and postgraduate
programs to include training in communication skills for future developers and
software engineers. Moreover, current staff would greatly benefit from short-term
courses in how to communicate effectively. The latter has recently been considered by
OMII-UK however, it was found that there is no available funding for it, especially as
OMII-UK are at the end of their funding period.
Resources and Institutional Pressures
Limited resources such as time and staff effort have placed numerous constraints on
the work of OMII-UK. Beckles (2005) describes clearly the decisive role of software
engineers in e-Research software design and call for allocating funding for software
engineers. However, OMII-UK is based within the university system and its Full
Economic Costing environment does not allow for a proper recognition of the posts
of software engineers. In the words of an OMII-UK staff member:
“[Software engineers] cannot be funded at the same level as researchers. If we
receive a grant we can claim twice as much for a researcher than for a software
engineer and this is a severe constraint. The HR [Human Resources] policy says that
if you want to employ a software engineer you have to employ them as such but
they get less money [than a researcher]. The impact is that our group is seen less of
an asset to the School where we are based. HR departments are not aware of the
financial consequences of their policy. We try and reclassify software engineers as
researchers and this is preventing us from concentrating on the software as we have
to manage internal problems”.
Limited resources to dedicate specifically to software engineering impose strong
limitations not only on the quality of the software produced but also on the scale of the
projects that can be undertaken. Often, the software produced in an academic

environment tends to be more proof-of-concept and prototype than production level
applications. These are perfectly well suited for research purposes that investigate and
test the characteristics of specific categories of applications. Unfortunately, they tend to
lack many features that make them easily adoptable by potential e-Research user
communities.
e-Research communities tend to need robust, properly usable and well
integrated applications and a lack of usability represents a great problem for the type of
software that OMII-UK and similar institutions (help to) produce. As seen in the
previous sections of this report, the OMII-UK user community mainly consists of
researchers and research groups that need high quality software to conduct their
experiments and projects. In the word of an OMII-UK user:
“I have a team of researchers that need to do science and require to access
distributed resources. You give me a set of instruments to submit my jobs but some
of them do not work properly and others do not fit with my requirements”.
The effort to produce software for e-Research that can be properly adopted by its
intended user community involves the management of multiple layers of institutional
policies. The question of resources is not only about funding being available but also
about how it can be allocated in an academic environment. The development of an e-
Research community with all its connected development activities implies necessarily a
concrete process of integration of new research, development and support models into
pre-existing institutions, practices and policies. Without an accurate control, analysis
and assessment of such a process the rise and consolidation of e-Research will not be
possible.
Communication and the Role of Language
The widening of the user base to include users of different levels of computer literacy
has posed specific engagement challenges because each level of awareness requires a
different approach. Thus, user engagement necessitates more planning. It is important
that OMII-UK staff has paid attention to and is sensitive to the needs of different
users:
“The level of conversation on requirements is different with different people, for
instance, some [users who] are looking at language research […] have very different
perspective and they express the requirements in functionality in a different way.
[While some] express requirement as: «we must be able to define whether or not
software is robust after the result of refactoring» [others] express like: «I want to be

able to analyse my text ten times faster». There are different levels of vocabularies
and it is [necessary] to understand where the right level is to talk to them”.
OMII-UK has shown determination to establish effective communication channels
with users. For example, they had planned their Collaborators workshop in the spring
of 2009 as an avenue where they could listen to the users: “we wanted to talk about
what the users wanted to talk about” so they had a “flexible agenda” for the workshop.
Following the users‟ suggestion on the day, they decided to hold one of the break-out
sessions on “how to talk to scientists”. This session had about six people and lasted
about one hour and what emerged was the need for developers and users to be able to
spend more time together to begin to understand each other‟s language.
During requirements gathering and development OMII-UK employ different
means to communicate with users: “we found that the best way is to start with a set of
initial questions by email then face to face (one or more sessions). It is faster to achieve
the understanding face-to-face. After that, telephone or face to face supplementing
email”. However, the case might be that even more time for user engagement might be
necessary for OMII-UK if they wish to achieve their goal to “look at the software in
context and understand the way it will be used, really try and understand it”. At the
same time, all interviewees had agreed that this understanding is crucial for eliciting
requirements that guarantee usability.
Another set of issues related to communication and the use of language that is
directly linked to usability pertain to the production of documentation.
Documentation and Support
Documentation and other forms of support in the use of software are indivisible from
usability issues. As OMII-UK staff commented, the clarity of documentation has
indeed been an endemic problem among e-Infrastructure providers. OMII-UK have
made a shift in the way they prepare documentation since they have begun to engage
users in its production and hired a specialist technical writer to manage this process,
for example: “[we have added] an additional piece of documentation that goes beyond
what you normally have as documentation – why did we chose to do it this way – it
was based on feedback from previous users. They expressed a feeling to do some
tutorials as they learned how to use it by talking to developers but developers cannot
go around to everyone to tell them so we went to the first users to see what will be the
best way of structuring this documentation”.

The current OMII-UK‟s production of documentation is described by the
technical writer as “bilingual”: “one for the computer savvy and one for users who are
illiterate in the field. The problem is that computer literate people do not like it simple
and the others cannot understand the more complex. There may be a need to create
two sets of documentations”. However, the production of such quality of
documentation is more costly: “this instantly doubles the time and money spent”.
Often documentation is scant, it lacks an introduction and a section on
installation. This is how the OMII-UK technical writer has solved this problem: “I
write an introduction – what the software does from the point of view of a user who
does not know so much – then I add a section on installation which is meant for the
developers. After that section, the computer literate could go away. The rest of the
documentation is for the end user. I do not use acronyms and I do not assume they
have the knowledge. At the end I give links to further information”. This shows that
preparing documentation, that is communicating the software to the user, is not to be
underestimated as a specialised professional activity and it is best done by professional
technical writers in consultation with the users and may need to be specially budgeted
for. At the same time, although documentation should be taken seriously, it must not
be used to compensate for poor usability.
Support in the use of software is closely linked to issues of usability (see also
Beckles 2005; Ure et al. 2008) and, in the context of widespread poor usability, its role
is very important. For this reason, how effective OMII-UK is in offering satisfactory
and timely support determines how users get on with the use of software. OMII-UK
offers different types of support but the main one remains the online support desk
where users send an email which becomes a “ticket”. As staff members recognise, “it is
important to see the tickets and respond to them in a satisfactory manner. The end-
user is asking a machine to help and does not know what is on the other side and that
person needs to respond in a timely fashion”. However, responding to the tickets
effectively entails proper management of the process and the involvement of local and
distributed OMII-UK team members. This how one OMII-UK staff member
described it:
“We have a FootPrints [IT service management] that tells me when users call for
help and how long it is before anyone respond. A ticket arrives to an input queue
and two of us have the authorisation to modify it. When both of us are off things
can get delayed. I make an indication for internal communication [about] how

important this user is […]. You can have a severity and a priority: how badly it
affects the end-user and how important it is for our organisation. Normally all
tickets are equal unless we recognize one is special. We never downgrade although
we can upgrade. E-mail dialogue with the users is done through the system but it is
often done directly person to person. I cannot do much about it. It should not be
dealt with off ticket because the resolution is not seen”.
Although support issues may, at first sight, appear to concern only staff, in the long
run, they also concern the users and usability. Keeping track of tickets and their
resolution is important if the provider is to have a systematic understanding of what
the persistent problems with usability are and how to go about them. An improper
support management may also prevent the build of institutional memory about what
has been resolved and how. The following excerpt suggests precisely this:
“It is a problem in a university environment as anyone feels they can adopt any
method of communication rather than use a common one. In industry it is
mandated […]. We have discussed this problem at management meetings but you
cannot align management goals with individual goals in a university. Historically, the
OMII-UK sites have different mechanisms of dealing with user requests: in
Manchester, they have a mailing list; in Edinburgh, Bugzilla; in Southampton,
FootPrints. We do not have a way of integrating the support system as part of the
groups. This is a problem of multiple sites. So we accept that once a ticket is
assigned to a different site, they transfer it to their internal system and the end user
may be effectively dealt with but this information will not be captured by the
ticketing system. We cannot look at the totality of our support tickets and see what
goes on. In contrast, in Oracle every interaction with their customers is captured by
this system”.
Again, this situation points to the importance of viewing usability as a systemic
property of the entire organisation that designs, develops, provides and supports a
software tool. The internal and external dynamics of such an organisation have to be
shaped so to guarantee proper, bidirectional communication with the users. It should
never be underestimated the importance of taking into account the wider institutional
context.

Working with the Developers
A significant proportion of OMII-UK‟s users are developers. OMII-UK‟s work with
developers includes different stages and a variety of activities such as those on the
Commissioned Software Program (CSP). A recent example is the work done by OMII-
UK with developers on security and job submission and how to merge them into two
projects: an applications repository and the Rapid
9
tool for designing portal interfaces.
“Both included work to do with managed ways of transferring data and got together
and did some additional work in collaboration involving them and a group in
Australia and developers had gone there and Australian developers had come here
and they worked together and this also continued at the collaborators workshop. It
is an ongoing input to address usability. From developers point of view about
services that transfer large data from one site to another; developers are users
because they work with OMII to help develop things”.
When OMII-UK chooses which developers to engage with, it uses the following
criteria:
“The level of activity determines if this is a product that is it worth engaging with.
Generally, for an open source product, good indicators are: how many posts are
there in their mailing list; have they been answered; how often do they release their
software. Generally, pieces of software that are well supported will be released
regularly with bug fixes and new functionality. For Free Open source software there
is the issue of having a sustainability model. If you develop a project you are not
going to use a piece of software that is not well maintained showing that you have a
piece of software that is evolving and that has a future. Sometimes, you find bits of
software that do what you need and that are sufficiently robust so you do not need
to worry about new versions. If you are an end-user, you want something that that
is actively being developed. Sustainability ensures that it has a future”.
However, there may be space for expanding the engagement with end-users here in
order to understand whether and how the software is used in practice.
The process of software evaluation shows how OMII-UK addresses usability issues
through engaging the developers and their users. An OMII-UK interviewee reported
that:
“We have a collection of errors. We collect emails and then follow up by face to
face and this is not a single thing, it is a repeated process. Evaluation takes place in
9
http://research.nesc.ac.uk/rapid/

the process, it does not come at the end. Problems come through helpdesk and
email and [...] people come to us at workshops and they tell us. We make sure we
capture it and in the email we send them we summarize what was said and ask them
to clarify and then we follow up as a normal process”.
Apart from this more ad hoc and informal way of ongoing evaluation (as one of the
interviewees put it: “academic evaluation of software is not very useful and most
people in the e-Science program will say that there is a permanent state of evaluation”),
OMII-UK staff also relies on a more formalised evaluation process. This includes the
implementation and ongoing improvement of their Quality Assurance (QA) process in
accordance with the academic users‟ demands in a constantly changing technological
landscape:
“Our QA has its root in the original OMII 2004 when it was set up as an
organisation for evaluation of software developed by the community and the
original goal was to be capable to support it. […] This original process of
evaluation has changed: we do not look at the code but at the packaging and
documentation. Is it possible to download a piece of software in one go and in the
right format? You need clear listing of prerequisites. For example, operating
systems, browsers and other software that must be installed. Then we look at
installation (how to unpack) and configuration documentation (how to link it to
anything you need to link it to; customisation for your site). If you have this
documentation you may succeed but there is such a diversity of architecture in
universities so your chances of getting the right combination are quite low. […]
The availability of operating systems and browsers change at different rates at
unpredictable times and it is difficult for us to keep up to date with the operating
systems and middleware, including data bases. In academia people tend to use the
latest, but in a corporate environment they may fix their systems on one system
and may not change it - so if your users have an older version you have to do it but
in academia - the challenge is to support the latest - so we support the latest stable
version of things - e.g. Internet explorer 7 but not 8”.
The main goal guiding the way OMII-UK conduct QA is usability:
“QA improves stability and robustness and improves sustainability as you can set
realistic expectations. You manage what the quality level and what the support level
is. We aim to improve usability for both users and developers through QA […]. We
had a QA checklist and the good thing was it was easy for technical staff at OMII to

carry it but it was not suitable for different types of software, it was better for grid
and web computational service but not good for Portlets and Portals and client
applications. And the reason it was not good was that it was a set list. We could
have had a set list for each program but it is a lengthy process. The new QA is a set
of criteria. Rather than have checklist we have a set of categories and these
categories are broadly related to usability from a developer point of view. It was
done after we tried it on some pieces of software we evaluated and we ran a test.
We started with categories we had and tried to understand which one was relevant.
The old one was not working. The signs were that some of the software was taking
a long time to go through evaluation and we noticed that when we looked at the
answers they are either N/A or not of use to developers or end users”.
However, although this process was done from the point of view of end-users and
developers, actually, the checklist is filled in by OMII-UK staff. It remains to be seen
whether the actual involvement of developers and end-users will bring further changes
to the QA process.
Bringing the users together
Most recently, OMII-UK have begun to bring all different categories of users together.
For example, on the OSCAR
10
project they are collaborating with developers but they
are also planning to directly engage other end-users such as researchers in order to
elicit their requirements:
“OSCAR is [a tool for shallow, chemistry-specific parsing of chemical documents].
We wish to improve it. We had a meeting [with the developers after evaluation] and
we agreed on common issues and this was an iterative project. […] Some people are
using it and we plan to determine how they use it and what they want and how to
fix it for them. We will hold a workshop and we will bring the existing users
together to present the work we do and understand what they want from the
software and how they are using it. We are doing refactoring, taking it in its current
form and changing it in a form that is better suited for open source development”.
This process well summarise the ongoing effort at OMII-UK to improve usability of
their products by working not only with developers but also with end-users. In so
doing OMII-UK is proposing as a catalyser for the provision and support of high
quality tools for the e-Research community. An open challenge remains to tightly
10
http://www.omii.ac.uk/wiki/OSCAR

coordinate with institutions and national service providers that contribute to the UK e-
Infrastructure. If this collaboration will be conducted promoting a user-centred process
of e-Infrastructure provision the whole process that goes from the design of e-
Research oriented applications up to their provision through a national grid service will
guarantee a high degree of usability and a proper level of support.
For this reason, the next section of this report is dedicated to the analysis of
the data collected about the UK National Grid Service (NGS) with particular attention
to how their service provision meet the accessibility and usability criteria of the e-
Research community they serve.
The National Grid Service (NGS)
NGS was established in 2003 when it was transformed from a test-bed into a
production site. There were no NGS users until the end of 2004 as it was only
announced at the UK E-Science All Hands Meeting in 2004. It consisted of four sites:
one at RAL and three at the UK universities of Manchester, Oxford and Leeds. Every
site acquired a cluster for a total of around 1000 CPUs and several terabytes of storage
space. A grid framework, an authentication and authorisation infrastructure and several
scientific applications were made available on these resources to the UK e-Science
community.
The technical and organisational choices made when starting the NGS in part
shaped how the accessibility and usability issues would evolve in the following years.
Probably, the two most relevant choices were the adoption of a software stack based
on the Globus Toolkit
11
and the adoption of the X.509
12
standard for user and service
authentication and authorisation.
The Globus Toolkit is an open source project provided and maintained by the
Globus Alliance. The Globus Alliance is an international collaboration effort that
brings together many different resources, mainly provided by US and EU universities.
Although Globus has matured significantly since 2004, it still retains some of the
typical characteristics of an academic software project. For example, its design changes
frequently, there is often a certain degree of incompatibility between subsequent
releases, its installation and configuration require expert knowledge, and it is generally
geared towards computational savvy users.
11
http://www.globus.org/
12
http://www.itu.int/rec/T-REC-X.509/en

X.509 is a standard for a public key infrastructure that allows single users and
services to be authenticated by means of a pair of public and private keys via a
computational service. The public and private keys are given to the user by a
Certification Authority (CA) which assumes the responsibility to verify the identity of
the user and to maintain a secure copy of her or his credentials. While this system,
when properly administered, guarantees a high degree of security, it may be
cumbersome for users from a technical and administrative point of view.
At the beginning of NGS there was no centralised helpdesk or other form of
coordinated user support. The establishment of the Certification Authority (CA) dates
from that period and the lack of users at that time perhaps underlines the usability
problems posed by the certification process. As one of the interviewees said: “user
accommodation did not exist at the time”. However, gradually, NGS turned its
attention to the concerns of users and its internal organisation and practice evolved in
order to offer an NGS “seamless for users”. The idea was that “people [should be able]
to use it [NGS] on laptops and not to have to learn information technology to just use
more resources from desktop to supercomputer”.
In the process of NGS‟ expansion, the core sites were joined by partner and
affiliate sites. Partner sites were contributing resources while affiliate sites were able to
„try out‟ the NGS experience by having their users submit jobs to the NGS resources
without contributing their own resources. The core sites were still responsible for
running all the core services. Such services were managed by a coordinated pool of
dedicated system administrators and IT officers. This broadening of the NGS signalled
the beginning of a truly national service, not only concerned with providing
computational resources but also with creating the backbone for a national grid
infrastructure.
NGS received a second round of funding in 2006 and unofficially became
NGS2. This allowed the four core sites to renew their cluster infrastructures and
further expand their role at a national level. By 2008, there were “18 sites among which
7 partner sites and 9 affiliate sites, 800 registered users for core NGS Compute
Services, 5019 active certificates and more than 2049 user certificates” (Richards 2008).
The vision of NGS2, as described by Neil Geddes and Andrew Richards at the
NGS User Forum in September 2008, additionally focussed on the better integration
of all available resources to offer easier and wider access to individuals and academic
institutions. The core idea behind this vision was to make NGS the “glue” and

“gateway” to different segments of the national e-Infrastructure. The goal was to
provide an environment for collaborative research that guaranteed unified, nation-wide
access to computational and data storage resources beyond the specific differences of
local institutions. The methods for achieving this goal included:
1. Having Open standards;
2. Building value-added services;
3. Developing policies for operation, interoperation and sharing of computing
services across the UK;
4. Having a repository of user documentation;
5. Having a UK helpdesk and support centre;
6. Targeting training and outreach.
In April 2009, NGS received a third set of funding and unofficially NGS3 was born,
significantly changing its internal organisation and its middle term goals and long term
vision. The distinction between core sites, and partner or affiliate sites was abandoned
with the transformation of all institutions members of the NGS into partner or affiliate
sites.
Every institution contributes computational resources that are locally managed
and nationally coordinated. Thanks to these changes, NGS has become a streamlined
organisation that offers a heterogeneous set of services to the owners of a UK e-
Science certificate and to a pool of Virtual Organisations (VOs). From an operational
point of view, the current NGS pays close attention to the adoption of a common
software stack with production level services while offering verified documentation for
their installation and configuration.
Collectively, the NGS partners offer a vast range of services: certification,
monitoring, tools, portals and also commissioned services such as computing services,
data storage, visualisation, specialist training facilities, CSE type support, light path
connections and many others. The common software stack includes applications for
users and VOs authorisation and authentications, information systems, computing
resource services that allow users to run remote tasks on a resource, data transfer tools,
storage management, database hosting and user data access services, VOs management
and accounting, client tools, information aggregation, service monitoring, workload
management, user interface management, hosting and virtualisation.
The reorganisation of NGS3 should have a direct impact on the accessibility
and usability of the offered services. NGS end-users, mainly individual researchers,

should benefit from the stability and local support of production level service as they
now have improved documentation and more partner sites that offer resource pools.
On the other end, institutional NGS users, mainly Higher Education Institutions
(HEI), should find it easier to join and contribute resources to NGS thanks to
improved coordination among partners, a well-defined common software stack, a set
of verified installation and configuration documentation and dedicated staff effort to
the partnership process.
NGS3 is intended to support only HEI and other research institutions while
leaving the technical support of end-users to local resource providers. NGS3 will
gradually delegate the work with end-users to these providers during a period of two
years. So called “campus champions” will be identified who would offer technical
expertise and support to new NGS end-users.
Clearly, this process of localisation and decentralisation is potentially
detrimental from an end-user point of view. The risk is that NGS end-users will lose
the nation-wide support offered up to the present time by the four core sites without
having precise direction, instruments, procedures and policies for obtaining support
from the institution running the resources that they are using or from their affiliated
university. If NGS manages to establish such procedures and policies while offering
clear direction to its current end-users, its localisation strategy might be successful also
from an end-user point of view. Without this provision, users are likely to experience
frustration and may opt out of NGS services.
In order to avoid this problem, the process of localisation will be gradual and
NGS continues to offer end-user support during this transition period. Among the
type of users with whom the NGS will aim to continue direct involvement there will be
not only HEIs but also large public (and possibly private) research groups and
institutions. An example of such users is the research group involved in the GENIUS
13
project run by Peter Coveney at UCL. The GENIUS project adopted HARC
14
, an
advanced reservation system, collaborating with the NGS staff at Manchester, Leeds
and Oxford. NGS has helped GENIUS by debugging HARC and deploying it on their
resources. Also in this case, the collaboration process has maintained a user-centred
approach. As one NGS staff puts it: “the only way to engage with the users is to
understand them. We cannot go away completely from the end-users”.
13
http://wiki.realitygrid.org/wiki/GENIUS
14
http://www.cct.lsu.edu/harc.php

Currently, end-users come to NGS for a variety of reasons but most report that
they use it because it is free of charge; institutional users are mainly interested in being
part of the grid infrastructure and obtaining the NGS brand. NGS users fall within
three categories:
1. Expert users with advanced technical knowledge;
2. Users who do not know much about the grid or how it works; and
3. System administrators or IT officers.
In the following section key examples illustrating how NGS has engaged its users to
the present time are identified. Further mention is made of measures to ensure
continued accessibility and usability of the national e-Infrastructure and what
challenges NGS and its users have encountered in this engagement. Although NGS
will increasingly move away from catering to end-users in subsequent years, our
analysis and the recommendations that follow will still be valuable across the whole
NGS3 and the local providers that will take over the end-user engagement and support
process.
User Engagement
Since May 2007, NGS has focussed on user outreach with the goal to increase its user-
base and better serve its current users. This effort aims at improving the user
engagement process and the overall quality of the user experience. The user
engagement initiatives are:
 Two types of mailing lists, one for discussion and the other for critical
announcements about the status of NGS resources;
 An ongoing rationalisation of the NGS website;
 The creation of a newsletter published quarterly including news about events
and conferences, the success stories of researchers using NGS, important news
about NGS resources and useful technical insights on how to use them;
 Outreach material such as case studies, posters and handouts are distributed at
events or published on-line;
 Conference attendance with dedicated NGS stands offering face to face
introduction to NGS, on-site user support and presentation material;
 Road-shows aimed at increasing the local knowledge of NGS in UK;

These initiatives have been backed up by training certified roaming operators capable
of requesting user certificates at conferences and events. This initiative is important
because it allows the users attending the events, to completely avoid all the described
issues associated with the certification process. Within minutes, users may have their
certificates downloaded on their USB pens and immediately access NGS resources.
Often, roaming operators may also give useful indications about how to use the
resources for the user‟s specific needs. Overall, this initiative has been successful,
creating a relevant number of roaming certificates between 2008 and 2009.
These engagement initiatives have enabled NGS to successfully communicate
with users face-to-face, particularly at meetings such as the Innovation Forum in
September 2008 or the recent road shows organised by NGS at different universities
across the UK: “This is about initial engagement […] and they [users] start to
understand what we do ad appreciate the technical challenges that exist and this can
help the local user to be more engaged”.
NGS sees these meeting as a golden opportunity to understand users‟
requirements as well as to create the right expectation level by describing what NGS is
realistically about. Thus, the initial engagement also serves as expectations management
and a two way process of communication. The information collected, especially about
user requirements, is reported to NGS management and operation boards thereby
guiding their strategic and technical decisions.
Apart from listening to users and talking to them formally or informally, NGS
have also conducted user surveys, both hard copy and electronic. These instruments
may offer useful indications about users‟ needs, opinions and requirements but they
might offer partial answers to the broad questions about users‟ real needs and the best
way to provide them. For example, possible issues associated with surveys could be
related to the sampling methods of such surveys and to whether they are sufficiently
indicative of the targeted NGS user communities. Other issues could concern the
quality of the collected data, especially whether the gathered user-views are substantive
enough. Finally, the „non-use‟ might be difficult to assess through a survey as it is hard
to collect data about how an infrastructure is not used.
In the context of the recent refocus of NGS engagement and support activities
from single end-users towards HEI, research institutions and even commercial
organisations, NGS should probably consider every institutional user as a single,
unique case with particular characteristics. NGS should develop a tailored engagement

and support process that avoids the employment of a single solution for every
institution. As seen in the previous section of this report, OMII-UK has successfully
followed an analogous paradigm with the development of portal solutions. They have
focused on the requirements of a specific user and, once a viable solution has been
created, they have moved on to the following user utilising as much as possible of the
work already done but avoiding the creation of a single portal that tries to fit the needs
of every user.
It is important to note that NGS did not limit its effort at understanding user
needs to electronic or paper-based surveys. NGS outreach personnel collaborated with
three other JISC funded projects aimed at creating a better understanding of the e-
Science user community. These projects are ENGAGE, eIUS and e-Uptake. Due to
these collaborations NGS has refocused its effort in order to target specific user
communities and research groups with tailored service solutions. This effort is
consistent with our recommendations and should be adopted by NGS in the future.
User Experiences with Certificates
The problem with certificates is one of the best-known usability challenges for the
NGS. User certification is a cumbersome process demanding a lot of effort from users
in order to receive and install the certificates. NGS is aware of this problem. In the
words of a NGS staff member: “Certificates are a necessary evil. They are very flexible,
secure and, basically, unavoidable. Yet, they require a complex organisation to work
properly and, as usual with security, they impose tradeoffs on usability. Fortunately,
there are some ways to alleviate these shortcomings. They are technically difficult to
implement and they may impose a demanding overhead on the institutions but we
believe that they pay back with interests on the usability matter. This is why NGS is
trying as hard as possible to adopt Shibboleth even if inertia is difficult to overcome”.
Shibboleth is an authentication and authorisation infrastructure that would
allow members of UK institutions to log into NGS resources via their local credentials.
This means that, for example, a member of the University of Oxford would be able to
log into the RAL cluster using her or his Oxford login and password. While this
solution greatly improves the user experience, it clearly shifts all the complexity toward
the institutions that would have to federate in order to share authentication details.
This complexity, alongside those related to the implementation of the Shibboleth

technical infrastructure, makes the deployment of this system demanding for the NGS
staff:
“There is no time and to be honest. At the end of the day, we have already many
issues with the authentication infrastructure – I mean, just look at what happened
with the VO implementation; the recent issue with the proxy certificates; the
problem with the ssh keys and the CA... I don‟t hear news about the NGS
shibboleth implementation since almost a year. It is a pity, I mean, a lot of work has
been done”.
Until the NGS Shibboleth implementation becomes operational and improves the
current certification process, these problems will persist. As it is today, certification is
costly in terms of time and manpower for both users and the NGS staff members. The
following two user experiences illustrate this problem more clearly. The first excerpt is
from an interview with a computer specialist who describes his experience about
obtaining and installing a user certificate. The excerpt shows how the user struggled
with the international agreements about certification. The second excerpt is from an
interview with a researcher describing the frustration with the process of obtaining a
UK e-Science certificate.
1. “In 2003 I got a certificate. It was a difficult process. […] I was in Austria and
there was no Austrian registration authority. In the UK, they would not issue a
certificate for me while I was there. Today, Austria is part of the agreements
but it was not then”.
2. “I have been told to use NGS to submit my jobs. I went on trying to
understand what I had to do and after a while I understood that the first step
was to obtain a certificate. I filled in the on-line application but then I realised
that I had to go to find someone showing my passport. It really seemed a bit
over the top... Anyway, I went there, I showed my passport, I choose my
passphrase and all that stuff. I thought that was it but not really. I had to wait
for the certificate to be approved, I had to make a case for my research [...]. I
got that mail telling me that the certificate was ready and nothing happened. I
clicked on the link and nothing. At that point I was starting to be really
frustrated then I kind of remembered that the browser had already stored my
certificate. From there, I had to back it up and then to convert it into the
proper format. Fortunately that was a matter of cut and paste some commands

from the web-site. Anyway, the whole process was time consuming and
somewhat annoying”.
Both excerpts clearly illustrate the complexity related to obtaining an UK e-science
certificate. This is usually the first experience as a NGS user, something that risks
discouraging the user from the time of enrolment. Some NGS partners try to alleviate
the whole process with dedicated user support, taking away some steps of the process
from the user. This is clearly a good initiative that should probably be adopted by all
NGS partners.
The second excerpt highlights the importance of good documentation. The
user seems to undergo a kind of „trial and error‟ process in which he is faced with time
consuming operations that he was not necessarily expecting and that he feels are
somewhat unrelated with his need to perform his research. Precise, concise and
effective documentation can, if not reduce the complexity of the certification process,
at least make very clear what should be expected. Proper technical documentation is
also paramount, especially when it is necessary to use a command line interface.
User Experiences with Middleware
The difficulty with installing the Globus toolkit has emerged from our interviews as a
challenging experience for different types of NGS users. NGS manages this difficulty
with dedicated user support (both for end-users and institutional users) and by offering
a web portal from which end-users can submit their jobs to the available resources
without installing dedicated software on their own workstations.
One of the interviewed end-users, a biologist, clearly pointed out the
importance of web-based tools capable of hiding the complexity of command line
interfaces: “I know nothing about UNIX or how it is called. I never used it. I am a
biologist and I need to run my simulations. I use windows. We want select the
application we need, choose our data files, possibly manage our workflow and get back
the results”. The maintenance and further development of the NGS web-portal for job
submission should be one of the NGS‟ priorities in order to support those research
groups that do not have specialised IT competences. During our interviews, NGS has
confirmed the availability of dedicated staff effort for such a task.
When the portal is not a viable or desirable solution, NGS staff members tend
to closely support their users by implementing a process of handholding or, when
possible, directly stepping into the process of installation. A NGS user remembers that:

“I downloaded Globus. I was used to using computers but not certificates or
Globus and I did not know how everything is integrated there. I had to spend time
to try and it did not work. I emailed the systems administrator and we ping-ponged
for weeks to sort it out and since then I have not changed it. I copy it without
touching it. It seems that there is a new Globus version. I want to update it but 2-3
hours is a lot of time to spend for me”.
A NGS system administrator confirmed that:
“When appropriate, I personally install the Globus client toolkit on the user
workstation. I use the VDT
15
distribution and I spend around fifteen minutes to get
the client up and running. In my experience, this is way less than what it would take
me to help the user to install it via mail or at the phone. Some users have no
problems but when they do not care about knowing how it works, there is really no
better way that doing it myself”.
The importance of proper, step by step documentation clearly also emerges in this
case. The installation of the software needed to use NGS services should be as simple
as possible. Very often this is the second step undergone by new users in order to use
NGS. After the effort spent in getting a certificate, the second step should be as
seamless as possible in order to guarantee trust and appropriate expectations.
Overall, NGS staff members have acknowledged that it is a challenge to keep
on top of documentation. Documentation at the NGS is produced by NGS staff
members, mostly in their spare time. Feedback on the quality of documentation is
received by users that directly contact NGS because they “do not understand how to
use it [documentation] and when they have questions”. The documentation is
sometimes not easily accessible on the NGS website, on a wiki or via the help desk
and, often, it may be too complicated or lengthy. Presently, the production of
documentation is not seen as a two way communication and users have not been
consulted in the process of its preparation.
NGS3 has indicated a change of direction in the documentation production.
Consistently with the move towards the support of only institutional users – HEI,
research groups, private organisations and so on – NGS has dedicated six months to
the production of verified documentation for the installation and use of the
applications that compose the NGS software stack. This appears to be an important
15
VDT stands for Virtual Data Toolkit. VDT has been chosen by the NGS to offer a pre-packaged
distribution of the Globus Toolkit alongside other grid software. The use of VDT should make the
installation of the NGS software stack easier.

effort to improve the usability of NGS software stack but two challenges remain. First,
as with any other process addressing user needs, it should involve both the NGS and
its users. This involvement should include requirement elicitation, verification and
revision phases. The second challenge is to integrate the process of documentation into
all other NGS activities. In the absence of dedicated staff, it is difficult to produce the
required high quality documentation just in the „spare time‟ of the NGS staff members.
Partnership Joining Experience
Partner sites are a paradigmatic example of NGS institutional user. The process of
joining NGS is critical because, especially in NGS3, partner sites offer the bulk of
available computational and data resources. More partners generally mean more end-
users for NGS and, consequently, better integration within the existing UK e-Research
community. For these reasons, NGS needs to optimally streamline and execute the
partnership joining process.
The joining process presents technical and organisational challenges for both
the NGS and the partner sites. The NGS software stack alongside the authorisation
and authentication framework and the accounting systems have to be integrated into
the partners‟ resources while support and management procedures must be adapted to
the local staff effort and established best practices. Altogether, these challenges require
a notable commitment from the partner sites and the success of the joining process
may largely depend on how it is managed by the NGS.
The following excerpts illustrate the technical challenges faced by the partner
sites‟ system administrators when joining NGS:
1. “Departments underestimate the effort necessary to run a cluster. I have
worked with one of the universities and had really bad problems. They could
not understand my problem at all. I was collaborating with colleagues in their
university and we wanted to work together. I agreed to use their resources and
we could not install our code. We spent a lot of time trying. The support there
was lacking expertise. So, we then applied for a grant for HPCx”.
2. “We joined when I was available in the department to set it up and the
department were ready to join. I started working on the joining in March and it
took three months to set it all up. I installed all software and machines”.

3. “The installation takes a lot of time. [...] Each HPC is different and different
problems arise. To benefit, we also decided to install it on another machine –
while the first one, it took me 2 months; the second one took only 2 hours”.
Knowledgeable, rapid and dedicated support is crucial for the partner joining process.
In an environment in which NGS is potentially not the only provider of grid resources,
user satisfaction is vital to guaranteeing successful partnerships. NGS offers highly
qualified, professional expertise to every joining site. Particular attention has been
dedicated to developing scripts for the installation of the Globus Toolkit middleware
and Virtual Organisation (VO) support and accounting. This effort has been explicitly
acknowledged in the partner sites interviews:
“The people who helped me were Steven Young and Jason Lander. Jason did a lot
of work on VDT. Before that, the issuing of the certificate, it took 2 weeks. I
emailed and spoke on the phone with the support at NGS. We emailed initially to
get the background things, then 4-5 times we spoke on the phone, the longest
conversation was about 4 hours and the shortest about 30 min. The Sungrid Engine
causes a lot of problems. In the end, I deleted a few things from the code, which the
sys admins did not like but it worked, I then fixed it. The support scripts for VDT
are not well written and Jason rewrote the main VDT script. One needs a lot of
expertise to do this”.
The partnership joining process is not only a matter of technical difficulties and
support but also of political and social mediation. In order to better manage these
aspects of the partnership program, NGS offers both partnership and affiliation
opportunities to the UK HEI, research and private institutions:
“Our University wanted to affiliate with NGS. Some of the internal discussions
included different views, e.g. some said it might be political or not useful, some
suspected it of free loader: people who may be using [for free] other people‟s
electricity. Our university had experience with the Grid. It looks good for university
members to be members of the NGS and if your researchers need resources, they
can use them, if you are under capacity, you can use ours”.
The affiliation program is important because it allows the joining institutions to
gradually integrate within NGS, dedicating a slowly increasing amount of staff effort
while retaining full control over their resources until policy and political matters have
been addressed.

1. “Because we are affiliates, we can control who is to use the machine. We did
not have permission to be a full member from our central offices. It was not
clear to us whose responsibility it was to arrive at this decision. I did want to
install us as a full member but I got word to do it as affiliate only. Since then,
there have been a push to join as full member and how to change it – it is very
easy”.
2. “As an affiliate sys admin you manually pair an external user with a local
account of HPC. This means that the permission relies on sys admins. But if
the university is a full member, the full user list is updated on daily basis and it
maps on local machine and anyone can log on and use our HPC. A member is
supposed to publish a list of services on a local machine. That is something the
university were not prepared to do – but was it that could give authorization, it
was unclear”.
These excerpts show that there are many different reasons to join NGS: the necessity
to access more resources, exploring political or branding opportunity, but also the
growing necessity to satisfy the specific requirements of e-Research activities. Such
research is characterised by a high degree of distribution across different institutions
and by the necessity to coordinate the available computational resources in a national
infrastructure.
“Some researchers want to collaborate, some of them are on campus, others are
from elsewhere, some of the external students want to collaborate with our
students. We could also connect with US and Europe once we had Globus”.
NGS is already a primary actor in the creation and maintenance of a UK national e-
Infrastructure. NGS has accumulated many years of technical expertise in grid
deployment, organisation and management and it has already a large amount of
partners and affiliates. Our fieldwork clearly shows that, in order to be successful in
assuming a leading role in the UK e-Infrastructure, the NGS has to focus on both the
technical and political challenges of its partnership program. Adequate staff effort
should be assigned to this program and technical decisions should be prioritised to
make the joining process as efficient as possible. Specifically, the success of the NGS
effort in this context should be measured by the reduction of the time necessary to
become a partner site and by the increased capability of NGS to manage the political
implications of its partnership program.

Service Deployment Challenges
The Integrative Biology
16
Project (IB) was developed between 2004 and 2008 and it
was EPSRC funded. Seven UK Institutions were involved in the project: the
Universities of Oxford, Nottingham, Leeds, UCL, Birmingham, Sheffield, CCLRC,
IBM, and the University of Auckland. The IB project aimed to provide better
understanding of the causes of cancer and cardiovascular disease through mathematical
and computational modelling and to build the grid infrastructure to support this
research.
The IB project required the development of graphical user interfaces and
collaborative tools. The main challenges in developing these tools were both
organisational and technical. The organisational challenges included:
 The limited contact that developers and users had for many months at the
beginning of the project;
 The ongoing lack of regular communication between users and developers;
 The distributed nature of development;
 The distributed location of users; and
 The large variety of requirements from a large variety of national and
international users.
These challenges led to the slow development of the project and to some
misunderstandings between users and developers, which hindered the final outcomes.
However, the willingness on the side of both users and developers and the successful
facilitation by the project manger (who acted as translator between the two) eventually
led to the development of a tool useful to the users: “They [the developers] came often
– and would come back, e-mail, telephone, they preferred coming here. At one point
they understood the problem, our problem…” and the tool for visualisation was
developed.
One of the users described how the process involved the following aspects: “I
had to download on my machine all data and this was a big deal and they developed a
technology to visualize without this long process. They were also interested as it was a
challenge and integrated it also in a web service that could be accessed from one portal.
The software had bugs but it was a nice tool. IB was aiming at scientists without
experience and portals are good for it – people do not need to know. In my case – I
16
http://www.integrativebiology.ox.ac.uk/

am used to use Linux – you can have people with a lot of different kinds of expertise.
With a portal you can be anywhere – it is easier, we will use it”.
The technical challenges of IB were related to the reuse of e-Science
middleware and some specific issues concerning the applications: they used the Globus
Toolkit and “it was clunky… we were stuck with it and it was not fit for purpose and
we built our software stacks on top of it… it was built on a middleware which is
obsolete… we did used it because NGS were using it”.
Some users also thought that NGS may be crowded: “NGS is unusable, it is so
crowded – it may take 1-2 days to execute a job – so I spend time waiting, no point,
too many people using it. I now use HPCx resources directly”.
The fieldwork conducted within the IB project shows clearly that from an e-
Research point of view development and service provision need to be coordinated.
While e-Research has grown to a mature research field with many projects in mature
and late phase, e-Research is still developing and evolving its tools and paradigms. In
this very dynamic and fluid phase, middleware becomes easily obsolete, new projects
need large amounts of dedicated software development and production-level services
need to evolve accordingly.
As seen in the section dedicated to OMII-UK, user requirement elicitation and
management is crucial in order to accommodate the needs of the e-Research
community. The same is valid for the services provided by the NGS. Our fieldwork
has provided evidence of how the NGS had developed dedicated service solutions for
some of its institutional users. The NGS underwent processes of requirement
elicitation and iterative service development analogous, even if on a much smaller
scale, to those seen within OMII-UK. For example, one of the former NGS core sites
has collaborated with a user in order to offer him a cluster development infrastructure
for his project:
“This user needed a production cluster with hundreds of CPU in order to develop
his software. This is not a small requirement, especially when the cluster in question
is a production machine. We went through a fairly accurate requirement analysis
with the user. We had meetings, he presented his project, we discussed his design
from the point of view of our infrastructure and he amended it accordingly. After a
couple of months we realised that we would have been able to give him access to
the cluster without compromising its security and stability. That was a success story.
Since then, the user has developed his software in a way that is inherently

compatible with NGS. This is a big plus because it means that all his future users
will have little or no difficulty to use NGS. Basically, the learning curve is flattened
for free”
In another case, NGS had the opportunity to dedicate a specific amount of resources
to a single institutional user. Also in this case, requirement engineering and the iterative
deployment process were essential:
“They went to us because they knew we had a large amount of storage space
available. I mean, something like 4TB on a single partition. They needed the space
like yesterday so we went on with the partition. They were Linux-savvy so we
simply mounted the partition on one of their home directories and they were able to
fill it in immediately. Meanwhile, we went on with the requirement elicitation. That
was difficult. They change their requirements many times so we had to slow down
waiting for the situation to be clarified. Basically, we stalled for around six months
then, once clarified the situation, we went on implementing the infrastructure they
needed. We gave them a virtual machine with a hook on their partition and they
were able to go on from there”
These two examples clearly show the need to integrate both development and service
deployment into the same solution for e-Research requirements. Neither the NGS nor
OMII-UK have the necessary staff and resources to singularly put this integration into
practice. Such a goal could be achieved by their closer collaboration and coordination.
Their recent collaboration on the deployment of the OMII-UK GridSAM job
submission interface as a NGS service seems to suggest that moving in this direction is
viable:
“We have a portal that allows to submit jobs described in JSDL but we had at least
two research groups that needed something at lower level. We deployed GridSAM
but it had a bad reputation. Previous experiences ended up in a big mess [...].
Fortunately, we met some OMII-UK guys at a conference and they were willing to
collaborate. They offered high quality support to our admins and the service went
up. In the process, a couple of bugs were addressed, new realises were sent out and
our service reached in a relatively short period a decent stability”.
Conclusions and Recommendations
Our report, similar to previous research, has demonstrated that effective
communication and collaboration are integral to ensuring accessibility and usability for

both software and services. User engagement and support have proven to be of
paramount importance in order to guarantee proper requirement elicitation, service
customisation and effective strategic decision.
Our fieldwork shows that both OMII-UK and the NGS understand that user
engagement and support are important at all stages. Across the subsequent funding
rounds, both providers have changed their vision and practice in order to increase the
contact with their respective users. Both OMII-UK and NGS have specific staff effort
dedicated to user engagement and, overall, the quality and opportunities for
communications between providers and users have steadily increased.
It remains to be seen whether these changes will be capitalised upon.
Communication and adapting practices alone are just an instrument and the real
challenge is to implement these flexibly and widely and thus to provide usable services.
Our report shows that there are indications that such a challenge is still open for both
providers. Some of the interviewed users have expressed a certain degree of frustration
with the quality and accessibility of the software and services provided by OMII-UK
and NGS. This is not to say that progress has not been made, or that there are not very
satisfied users but the road towards a production level, fully usable UK e-Infrastructure
still seems far off.
Another clear outcome of our report is the need for both providers to increase
their collaboration. As explained in the introduction, a national e-Infrastructure is
made up of many different services and software and this mandates constant tailored
development. OMII-UK and NGS are fully complementary to each other, the former
providing support for development of e-Research tools, the latter providing support
and coordination of the services necessary to conduct e-Research. The recent
successful collaboration between OMII-UK and NGS is a clear indication of how
fruitful a closer and more coordinated collaboration between these two providers could
be.
Clearly, collaboration between service providers and users requires staff effort.
The right amount of resources should be allocated to the critical task of overcoming
the existing institutional barriers and engaging directly with individual end-users. It has
been pointed out how difficult it is to integrate engineering staff effort into academic
practice and assessment procedures. It should be considered a priority for OMII-UK
and NGS to devise an efficient strategy to guarantee the continuation of their activities
within an institutional framework.

The presence of e-Science and e-Research centres in the UK is a possible
answer to such difficulties. E-Research requires interdisciplinary projects that go
beyond the canonical boundaries of departmental subdivisions. E-Research is
conducted by researchers with a heterogeneous pool of competences that need access
to many different type of resources. Engineers, developers, experts in human computer
interaction and computer supported cooperative work and project managers are all
examples of professional figures that have to work together in order to make e-
Research projects successful.
Both OMII-UK and the NGS have increased their effort to avoid
fragmentation by consolidating those services and practices that have proven to be
necessary to the e-Research community. Our report clearly underlines how both
providers have endorsed flexibility as one of their primary characteristics. The NGS
has avoided committing to a specific middleware or to a restricting vision of grid
services. Analogously, OMII-UK has reorganised itself in the directions of supporting
targeted development for e-Researchers avoiding the provision of solutions from the
top disconnected from the users‟ real needs. This flexibility should be counterbalanced
by a crystalline vision and strategy. It is imperative to avoid offering numerous
alternatives solutions that have not been developed to production level, that do not
respond to concrete needs or that do not properly integrate within the strategies,
resources and policies of local providers.
Expectations and trust are essential for e-Infrastructure providers. Both OMII-
UK and NGS need to increase their user-base in part by avoiding loss of their existing
users. Our fieldwork has shown the fundamental role played by user engagement and
support in order to achieve these goals. User engagement should create the right
expectations about what the service providers offer while user support should
reinforce trust by helping new and existing users. The choice of OMII-UK to employ a
professional writer for their documentation and the NGS‟ decision to employ
dedicated outreach personnel are good starting points. A further step is investigating
how to capitalise on the body of existing knowledge in the fields of computer
interaction, computer supported cooperative work and requirement engineering. Many
problems connected to accessibility and usability have been studied and addressed in
such disciplines and it is possible that they could inspire efficient strategies and
methodologies for OMII-UK and NGS.

E-Research is a broader category including not only the, by now traditional,
scientific disciplines directly connected to computing – bioinformatics, computational
chemistry, astronomy, physics and so on – but also humanistic disciplines that, as
shown by NGS usage statistics, are starting to leverage the power of distributed
computational tools in their research. It is important for both OMII-UK and NGS to
understand the peculiar differences and requirements of this new category of users.
Their successful embracement and development of e-Research tools and services is
fundamental to confirm the necessity of general purpose service providers such as
OMII-UK and the NGS. The handholding of such users is made particularly difficult
by the distance that could exist between their cultural background and that of the
service providers‟ staff members. In this context, the effort shown by OMII-UK to
select developers with particular communication skills in order to interact with users
should also be taken into consideration also by the NGS. Scientific advisors with a
heterogeneous pool of competences could play a fundamental role in connecting both
service providers with their diversified user-base.
The decentralisation strategy adopted by NGS that aims to outsource the end-
user support to the partner sites opens interesting questions. From one side, it is
important to recognise the necessity to devise ways to integrate e-Research oriented
services into the IT infrastructure already existing in the majority of UK educational
and research institutions. Conversely, it is fundamental to realise that this integration
will only occur with very careful consideration and proactive management of many
types of issues. Technological drives can be constrained and/or enabled by political,
social and economic considerations based on the characteristics of each local HEI.
This is a substantial endeavour that will require very clear strategy and dedicated staff
effort.
Both OMII-UK and the NGS are by now well established service providers
with many years of activity. The same can be said for the UK e-Research effort as a
whole. During these years, a great deal of knowledge has been accumulated both about
how to do computational assisted research and on how to provide appropriate support
services. Our report has underlined the importance of this knowledge for new users
and for the accessibility and usability of a national e-Infrastructure. Both OMII-UK
and the NGS must devise appropriate ways to ensure that this know-how and body of
expertise is properly codified and disseminated.

It is also clear that users are a fundamental resource for the service providers
of a UK e-Infrastructure. Users are not only the source of requirements and service
adopters but also proactive sources of knowledge and help. Their creative use of
resources and research networks has to be taken into account and supported where
appropriate in making provisions and engaging them. Users are valuable sources of
support to other users and user networks should be encouraged and supported by e-
Infrastructure providers. OMII-UK and the NGS websites provide good home bases
for the promotion of an on-line, UK e-Research community. Such an effort would
effectively integrate the ongoing activities promoting user forums, mailing lists and
newsletters.
A concise numbered list follows with our suggestions to improve the
accessibility and usability of the services provided by OMII-UK and the NGS. Some of
these suggestions are sufficiently general to be extended to every UK e-Infrastructure
provider:
1. Users should be considered a proactive resource and put at the centre of
the software development process and of the service infrastructure design.
The best methods to do this are: user-oriented requirement engineering,
user-centred design methodologies, user engagement activities, user
support, user training and community building effort.
2. Better integration and coordination among the strategies of every UK e-
Infrastructure service provider. Specifically, OMII-UK and the NGS
should improve and tighten their collaboration in order to complement
each-other visions.
3. Communication skills and specific areas of competence should be present
in the OMII-UK and NGS staff. Such skills should be used to maintain a
constant, robust and reciprocal communication channel with user bases. In
this context, the availability of specialised scientific advisors should be
increased.
4. OMII-UK and NGS should offer production-quality core services. When
multiple services of the same type or with the same functional properties
are present, at least one of these services should be at production level.
Best practice for the utilisation and/or development of core services should
be clearly available and properly integrated into the user engagement and
support activities.

5. OMII-UK and NGS should continue their process of integration into the
HEIs thereby minimising the risks of fragmentation. It is fundamental to
maintain and promote a national e-Infrastructure capable of supporting the
distributed characteristic of e-Research projects. The challenge of balancing
local and national drives should be matched with dedicated policies and
training.
6. The knowledge and expertise accumulated by the UK e-Infrastructure
providers should be organically organised and properly disseminated.
Failure to do so could severely affect the UK e-Research program in case
of lack of future funding.
7. Staff effort and support should be reconsidered in order to achieve the
above suggestions. A direct and proactive engagement with users and
institutions is costly, but it repays benefits that make it highly worthwhile.

References
Barjak, F., Wiegand, G., Lane, J., Poschen, M., Procter, R. & Robinson, S. (2007).
Accelerating Transition to Virtual Research Organization in Social Science (AVROSS): First
Results from a Survey of e-Infrastructure Adopters. Paper presented at the 3rd International
Conference on e-Social Science.
Bechki, B. (2006) Gaffers, Gofers and Grips: Role-Based Coordination in
Temporary Organisations, Organisation Science, 17(1), 3-21.
Beckles, B., Brostoff, S., and B. Ballard (2004) A first attempt: initial steps toward
determining scientific users‟ requirements and appropriate security paradigms for
computational grids (2004). Proceedings of the Workshop on Requirements Capture for
Collaboration in e-Science, Edinburgh, UK, 14-15 January 2004, 17-43
Beckels, B. (2005) Re-factoring Grid Computing for Usability. All Hands 2005
Proceedings. http://www.allhands.org.uk/2005/proceedings/papers/565.pdf
Chin, J. and P.V. Coveney 2004 Towards tractable toolkits for the Grid: a plea
for lightweight, usable middleware. UK e-Science Technical Report.
Cummings, J. and S. Kiesler (2005) Collaborative research across Disciplinary
and Organisational Boundaries. Social Studies of Science, 35(5), 703-722.
Dutton, W. and E. Mayer 2008 – Ralph has this reference
Finholt, T. (2003) Collaboratories as a new form of Scientific Organisation.
Economics of innovation and New Technologies, 12(1) 5-25.
Galison, P. (1992) Big Science: the Growth of Large Scale Research, Stanford, CA:
Stanford University Press.
Geddes, N. (2007) Bulding a sustainable European Grid Service. Lessons learnt
from National Experience. Powerpoint Presentation, E-IRG Workshop, Heidleberg,
http://www.e-irg.eu/images/stories/meetings/2007-de/neilgeddes.pdf
Gibbons, M., Limoges, C., Nowotny, H., Scwartzman, S., Scott, P. and M.
Trow (1994) The new production of knowledge: the dynamics of science and research in contemporary
societies, Thousand Oaks, CA: Sage Publications.
Hardisty, A., Neil Chue Hong, Rob Procter, Lorna Hughes, Alex Voss and
Mike Fraser (2008) Birds of a Feather (BoF) Session on "e-Infrastructure: tool for the
elite or tool for everybody?, UK e-Science Technical Report Series,
http://www.nesc.ac.uk/technical_papers/UKeS-2009-01.pdf

Hey, T. and A. Trefethen (2003) Science and Its Implications, Philosophical
Transactions of the Royal Society London A: Mathematical, Physical and Engineering Sciences, 361,
1809-1825.
Jeffrey, P. (2003) Smoothing the waters: Observations on the Process of Cross-
Disciplinary Research Collaboration, Social Studies of Science, 33, 539-562.
Jirotka, M., Procter, R. Rodden, T. and G. Bowker (eds.) (2006) Special issue –
collaboration in e-research, JCSCW 15(4).
Kalawsky, R.S., Holmes, I.R., O‟Brien, J., Lewin, M., Armitage, S.,
Goonetilleke, T.S., and S.P. Nee (2006) Human Factors Audit of Selected e-Science
Projects, JISK Report, http://syseng.lboro.ac.uk/jisc/humanfactors/
Klein, J.T. (2004) Prospects for Transdiciplinarity, Futures, 36, 515-526.
Kling, R., McKim, G., and A. King (2003) a Bit More to IT: Scholarly
Communication Forums as Socio-Technical Interaction Networks, Journal of the
American Society for information Science and Technology, 54, 47- 67.
Lee, Ch., Dourish, P. and G. Mark (2006) The Human Infrastructure of
Cyberinfrastructure, CSCW 2006 Proceedings.
Lloyd, Sh. and A. Simpson (2005) Project management in multi-disciplinary
collaborative research. Proceedings of IPCC 2005, Limerick.
Newhouse, S., Schopf, J. M., Richards, A. and M. Atkinson (2007). Study of User
Priorities for e-Infrastructure for e-Research (SUPER).
Olson, J., Olson, G. and A. Zimmerman (eds). (2006) The collaboratories
handbook, Cambrdige, MA: MIT Press.
Richards, A. (2008) Current and Future plans of the NGS, NGS 2008 User
Forum Presentation, Manchester, UK
http://www.grid-support.ac.uk/content/view/263/120/
Ribes, David and Bowker, Geoffrey C. (2008) „Organizing for Multidisciplinary
Collaboration: The Case of GEON‟, in G.M. Olson, J.S. Olson and A. Zimmerman
(eds), Science on the Internet, Cambridge, MA: MIT Press.
Rixon, G. (2003) problems with Globus Toolkit 3 and some possible solutions,
AstroGrid Development Wiki
http://wiki.astrogrid.org/bin/view/Astrogrid/GlobusToolkit3Problems
Schroeder, R. and D. Spencer (2009) Social Scientists and the Domestication of
e-Research Tools, e-Social Science 2009 Proceedings, 24-26th June Cologne.

Sonnenwald, D.H. (2003) The Conceptual Organisation: an emergent
collaborative R&D organiational form, Science Public Policy, 30, 261-272.
Sonnenwald, D. (2006) Collaborative virtual environments for scientific
collaboration: technical and organisational design frameworks. In R. Schroeder
and A. Axelsson (eds.) Avatars at Work and Play, NY: Springer: Verlag, 63-96.
Ure, J., Rakebrandt, F., Lloyd, S. and A.A. Khanban (2008) Usability, the Tri-
Wizard Challenge: Recurring Scenarious in the Design of a HealthGrid Portal, HSI
Proceedings, Krakow‟08.
Voss, A., Asgari-Targhi, M., Procter, R., Halfpenny, P., Dunn, S., Fragkouli, E.,
Anderson, S., Hughes, L., Mineter, M., Fergusson, D. and M. Atkinson (2008)
Widening Uptake of e-Infrastructure Services. Proceedings of e-Social Science'08,
Manchester.
Voss, A., Mascord, M., Castelleiro, M.A., Asgari-Targhi, M., Procter, R., Fraser,
M., Jirotka, M.,Halfpenny, P., Fergusson, D., Atkinson, M., Dunn, S., Blanke, T.,
Hughes, L. and S. Anderson (2007) e-Infrastructure Development and Community
Engagement. Proceedings of e-Social Science'07.
Warr, A. Lllyd, S., Jirotka, M., De la Flor, G., Schroeder, R. and M. Rahman
(2007) Project Management in e-Science, Technical Report, Oxford University,
http://www.oerc.ox.ac.uk/image-library/Project%20Management%20Report.pdf
Warr, A. de la Flor, G., Jirotka, M., Lloyd, S., Schroeder, R. and M. Rahman
(2007). Embedding e-Science Applications: Challenges from the eDiaMoND Case
Study.
Technical Report, Oxford University.
Whitley, R. (2000) The intellectual and social organisation of the sciences, Oxford: OUP.
Wulff, W. A. (1993) The Collaboratory Opportunity, Science, 261, 854-855.