Content uploaded by Giuseppe Desolda
Author content
All content in this area was uploaded by Giuseppe Desolda on Feb 25, 2019
Content may be subject to copyright.
Enabling End Users to Define the Behavior of
Smart Objects in AAL Environments
Carmelo Ardito1, Paolo Buono2, Maria Francesca Costabile3, Giuseppe Desolda4,
Rosa Lanzilotti5, Maristella Matera6 and Antonio Piccinno7
Abstract
In Ambient Assisted Living (AAL), Internet of Things (IoT) technology is ex-
ploited to equip living environments with smart objects that communicate with the
outside world in an intelligent and goal-oriented manner and can support the oc-
cupants’ activities. Currently, providing such objects with new capabilities re-
quires several programming efforts. In this paper, we present an approach to com-
bine IoT technologies and End-User Development (EUD) paradigms and tools to
identify innovative scenarios where end users are directly involved in the creation
and customization of the AAL systems they use. We will present EFESTO, a Task
Automation tool that offers a visual interaction paradigm to enable end users to
easily express rules for smart object configuration, and discuss how the overall
approach can support daily practices of non-frail elderlies.
1 Carmelo Ardito
University of Bari, via Orabona, 4, 70125 Bari (Italy), e-mail: carmelo.ardito@uniba.it
2 Paolo Buono
University of Bari, via Orabona, 4, 70125 Bari (Italy), e-mail: paolo.buono@uniba.it
3 Maria Francesca Costabile
University of Bari, via Orabona, 4, 70125 Bari (Italy), e-mail: maria.francesca@uniba.it
4 Giuseppe Desolda
University of Bari, via Orabona, 4, 70125 Bari (Italy), e-mail: giuseppe.desolda@uniba.it
5 Rosa Lanzilotti
University of Bari, via Orabona, 4, 70125 Bari (Italy), e-mail: rosa.lanzilotti@uniba.it
6 Maristella Matera
Politecnico di Milano - Dipartimento di Elettronica, Informazione e Bioigegneria, P.zza Leo-
nardo da Vinci, 32, 20133, Milano (Italy); e-mail: matera@elet.polimi.it
7 Antonio Piccinno
University of Bari, via Orabona, 4, 70125 Bari (Italy), e-mail: antonio.piccinno@uniba.it
Ardito C., Buono P., Costabile M.F., Desolda G., Lanzilotti R., Matera M. and Piccinno A. (2018). Enabling
End Users to Define the Behavior of Smart Objects in AAL Environments. In proc. of Italian Forum on
Ambient Assisted Living (ForItAAL). To be appear in Lecture Notes in Electrical Engineering.
2
Introduction and motivation
Our current research aims at bringing innovation in Ambient Assisted Living
(AAL) contexts, with a specific focus on the elderly’s activities, by proposing new
approaches to build spaces equipped with technology for monitoring older people
behavior while fostering an independent lifestyle and health preservation.
The new approaches capitalize on years of experience on End-User Develop-
ment (EUD), a research area whose goal is to support non-technical end users (ore
end users for short) in the creation of products and services tailored to their needs
and desires [1-3, 12]. Specifically, our research aims to empower end users to co-
design, customize and evolve computer systems by flexibly composing services
and smart objects that are now available on the Web.
The design approaches and interaction paradigms we have been developing in
the last years may also be applied in Ambient Assisted Living (AAL) contexts.
AAL is a challenging application field for EUD. Specifically, AAL entails techno-
logical interventions in the home, the most intimate and personal of the living
spaces. Internet of Things (IoT) technology is exploited to equip living environ-
ments with smart objects that communicate with the outside world in an intelligent
and goal-oriented manner and can support the occupants’ activities [4]. Currently,
providing such objects with new capabilities requires several programming efforts.
A challenge of our research is to investigate how IoT and EUD can be combined
to identify innovative AAL paradigms where end users are directly involved in the
creation and customization of the systems they use.
Smart homes form networks between people and objects that can be
conceptualized as “information ecologies”. This concept stresses the situated en-
tanglement between people (their values and behaviors) and technologies (their
requirements and functionalities). Moreover, older age is a period of strong inter-
individual differences claiming for highly flexible and customizable solutions.
Despite a financial support of up to 700 million Euros from the AAL EU program
(2008-2013), little social and practical benefits emerged so far and population
aging is still one of the main challenges for Europe. Such failures have been asso-
ciated with a major focus on technological solutions and a substantial lack of un-
derstanding of older people, their evolving needs and desires.
The solutions proposed so far in AAL are still technology-centric and consist
of monolithic, rigid systems that cannot be adequately customized to support the
information flow needed for care provisioning. Moreover, elderly people can live
in very different situations and can have variable needs and behavior; thus a one-
size-fits-all system would not work. Also, current interventions addressing safe
and independent living have designed their deliverables around the dominant ste-
reotype of older adults as people in need, and technology as the solution to their
problems [21-23]. However, there is evidence that this stereotype leads to the
inappropriate design of artifacts, which older people may refuse to use, or, in the
worst case, may be hampered by [8]. More importantly, it may easily lead to a re-
inforcement of ageism and possibly discrimination.
3
Our aim is to empower end users and other stakeholders of AAL to co-design,
tailor and evolve the technology they use by flexibly composing services and
smart objects. The focus is on end users that base their daily practices also on the
interaction with the surrounding environment. Non-frail elderlies are in particular
addressed. Frailty in elderly people is a state of vulnerability with an increased
risk of adverse health outcomes, caused by the cumulative decline in multiple
physical and psychological areas [9]. We target “non-frail” elderlies who lie on
the borderline between “active” elderly people, who still maintain good relational
involvement and high motivation for an independent life, and “fragile” elderly
people, who start to perceive the physiological and cognitive decline in their capa-
bilities and feel insecure in their living alone.
Our research adopts a socio-technical paradigm claiming that technology con-
sists of social and technical systems interacting with an external environment
(where people live or work). We investigate how to design products and services
for and with older users. In this paper we describe how a platform recently devel-
oped can be used to support non-technical people to tailor AAL applications to
their needs.
A quick look at the state of the art
A number of European research projects have been recently funded in the field of
AAL. However, to the best of our knowledge, the proposed solutions merely ad-
dress technological issues (e.g., sensor interoperability) or some specific accessi-
bility aspects, while none of the projects faced with EUD challenges, as this pro-
posal aims to do. Actually, several AAL projects (see for example ALFRED
(alfred.eu/project), GIRAFF+ (www.giraffplus.eu), DREAMING (www.age-
platform.eu/all-projects/658-dreaming) and inCASA (www.incasa-project.eu))
identified the interaction paradigm as a critical factor for lay users. They, in par-
ticular, focused on either adaptivity or adaptability of AAL systems; however,
none of the solutions they offered came to completely solve the challenge of
spreading the adoption and the personalization of the new technology among the
end users.
An AAL environment for the elderly population we address is a smart home
where several smart objects are available. A smart object is a device equipped
with embedded software that is typically connected to the Internet [5]. It exploits
sensors to “feel” the environment/users and/or actuators to communicate with the
environment/users. Examples of sensors are those measuring light intensity, the
physical pressure of an object, and also air humidity. Actuators can be light and
sound emitters, electric valves, motor servos, relays, etc. Smart objects can foster
important changes in people lives and in particular in elderly lives. If enabled to
exploit the abundance of resources (the object functionality, the produced data, the
related applications), non-technical people could compose the “behavior” of the
surrounding environment to accommodate their everyday needs. However, while
4
research on the Internet of Things (IoT) [5] has devoted many efforts to the tech-
nological aspects characterizing smart objects, little social and practical benefits
have emerged so far. Programming the behavior of smart objects is currently a
prerogative reserved for professional developers, as it requires the use of scripting
languages that can also vary depending on the underlying hardware. Another as-
pect is that often the available objects expose a very specific functionality that
does not result in useful services able to accommodate users’ needs. The opportu-
nities offered by IoT can be amplified if new approaches are conceived to enable
non-technical users to be directly involved in “composing” their smart objects by
synchronizing their behavior.
Recently, some approaches have been proposed to support non-technical users
to configure the behavior of smart object behavior. Many such tools, however,
provide pre-packaged solutions, e.g., vendor- and device-specific apps for remote-
ly controlling single smart objects that cannot be easily adapted to the require-
ments deriving from specific domains and contexts of use. Task-Automation (TA)
tools that combine social services, data sources and sensors, are also gaining mo-
mentum [10]. They are gaining attention as they offer alternative paradigms to
synchronize the behavior of objects and applications [16]. In particular, users can
specify object behavior by either: a) graphically sketching the interaction among
the objects, for example by means of graphs that represent how events and data
parameters propagate among the different objects to achieve their synchronization,
or b) defining event-condition-action (ECA) rules [19], a paradigm largely used
for the specification of active systems (see for example [7, 11]), that in the IoT
domain can be fruitfully exploited to express how and when some object behav-
iors have to be activated in reaction to detected events. An example of graph-
based tool is Node-RED, an open-source Web platform to compose both smart ob-
jects and Web services [15] by wiring nodes representing smart objects, control
statements, functions written in JavaScript, and debug procedures. An example of
TA tool based on ECA-rules is IFTTT (If This Then That), a free Web platform
that, by means of a wizard-based composition paradigm, guides end users to create
simple chains of conditional statements called “apps” [14]. Each app consists of
(1) a service that IFTTT tracks to detect if a specific event is triggered (e.g., the
position of an Android Device is within a specific area) and (2) another service
that reacts to the triggered event by executing a specific action (e.g., switch on the
smart Vacuum Cleaner).
However, the adoption of TA tools is still limited. Indeed, on one hand tools
based on graph metaphors do not match the mental model of most users because,
as demonstrated in different works on visual service composition [18, 24], they do
not think about “connecting” services. On the other hand, tools implementing
ECA rules, which are typically suitable for non-technical users, allow a trivial
synchronization of smart-objects behaviors, without the possibility to define pow-
erful constraints on events activation and actions execution. For example, in [6]
authors discuss the importance of temporal and spatial conditions to create ECA
rules that better satisfy users’ needs. Specifying temporal conditions also emerged
as an important requirement in home automation, to schedule rules for appliance
5
activation [20]. Some of the available TA tools based on ECA rules allow the def-
inition of such spatial and temporal conditions, but only by means of worka-
rounds, for example by considering additional events to monitor the system time,
or by creating filters on smart device data (e.g., in Zapier [25]). Obviously, such
workarounds complicate the rule creation, thus resulting into a scarce adoption of
the available tools, especially by non-technical users, or in their adoption only for
very simple tasks.
At the IVU Lab we have implemented the EFESTO platform, which actually is
a TA tool that offers a visual interaction paradigm in order to enable end users to
easily express rules for smart object configuration that are more powerful than the
rules created by other TA tools like IFTTT. The paradigm is based on a model,
called 5W, which defines some specification constructs (Which, What, When,
Where, Why) to build rules coupling multiple events and conditions exposed by
smart objects, and for defining temporal and spatial constraints on rule activation
and actions execution. Such paradigm has been designed during an elicitation
study involving 25 participants [13].
Using the EFESTO platform in a AAL scenario
In order to describe how the EFESTO platform can be used in an AAL context in
order to allow end users to specify the behavior of smart objects, let us consider
the following scenario. Alfredo, aged 80 years old, lives in the city's outskirts and
is retired. In spite of his physical limitations, he lives alone because his home is
prepared to provide daily life assistance to an elder person. A system installed in
his house provides an environment with a range of interconnected sensors, devices
and smart appliances working together to provide a safe and secure place to live.
These appliances allow Alfredo an easy utilization due to their customized inter-
faces and are connected to the smartphones of Pedro and Joana, his children.
When Alfredo goes to the five o’ clock walk, his absence from home is detected
by the changing position of the smart bracelet he wears. Thus, the vacuum cleaner
robot starts its work. Later, Alfredo comes back home. At seven o’ clock, the
doorbell rings. The devices positioned at the door detects that there is nothing to
worry about as it is Alfredo’s sister, identified by her smart-watch. The door au-
tomatically swings open, as the house already knows she has permission to get in.
We now illustrate how EFESTO allows a person to create one of the rules in-
volved in the above scenario. In particular, Joana wants to create the following
rule: “turn on the vacuum cleaner robot when Alfredo’s bracelet detects that he is
outdoor for the five o’clock walk”.
On the main window of the EFESTO platform, Joana clicks the “New Rule”
button in the navigation bar (Fig. 1, circle 1) and the “Creating Rule” window ap-
pears, as shown in Fig. 1, showing the main area in which a rule is defined. The
left side is for specifying the triggering events, and the right side is to define the
actions to be activated by the appropriate services.
6
Fig. 1. EFESTO: the interface for rule creation.
To add an event, the user clicks on green “+” button highlighted by circle 2 in
Fig. 1. This activates a wizard procedure that guides Joana in defining the rule
event. Specifically, the wizard sequentially shows some pop-up windows in which
services, events and temporal and spatial conditions can be specified (see. Fig. 2a,
Fig. 2b and Fig. 2c). More in details, the user defines an event by selecting Which
is the service to be monitored for detecting the triggering event (Fig. 2a), What
service event has to be monitored (Fig. 2b), When and Where the event has to oc-
cur (Fig. 2c). Actually, the specification of When and Where conditions is option-
al. By referring to the example in Fig. 2, Joana by a simple click, selects the An-
droid Wear object (bracelet) in the window in Fig. 2a (Which) and the “Position
changed” event in the window in Fig. 2b (What). This because the event that acti-
vates the rule is that Alfredo, who wears the bracelet, is now outdoor. By operat-
ing on the window in Fig. 2c, she also specifies that the event activates the rule if
the Alfredo’s position is outdoor (Where) and if it happens in the time interval
from 5.00 p.m. to 6.00 p.m. (When). At this point the wizard procedure ends and
the defined event is shown in the “Events” area (Fig. 2d, circle 1).
After the event definition, Joana starts the creation of an action by clicking on
the “Add an action” button in the Actions area (Fig. 2d). Similar to what happen
for the event, the button activates a wizard that helps the user define an action in
terms of Which service will execute the action as a consequence of the event(s),
What action the service has to perform and When and Where the action can be
performed. In this scenario, Joana chooses the vacuum cleaner robot (Which) and
the “Start Cleaning” as action to be activated (What), without specifying any spa-
tial or temporal constraint. Joana can enrich her rule by adding further events and
actions. More details on this rule composition paradigm implemented in the
EFESTO platform can be found in [13] and will also be discussed at the confer-
ence.
1
1
2
7
a
b
c
d
Fig. 2. EFESTO wizard procedure for event specification: a) the wizard first asks to select the
service that will activate the rule; b) as second step, the event is selected among those related to
the chosen service; c) temporal and spatial constraints are defined; d) the event has been defined
and the user can define further events or actions.
3
2
1
8
Conclusion and future work
Population aging, the increasing cost of formal health care, the caregiver burden,
and the importance that the individuals place on living independently, all motivate
the development of innovative Assisted Living technologies (AAL) for safe and
independent aging.
The widespread of AAL applications will reduce admission to hospitals or
nursing homes, permitting a better quality of life and savings for the community
[17]. So far, several AAL projects have been proposed. While the technology is al-
ready well advanced, the major challenge for the success of AAL projects comes
from the Human-Computer Interaction (HCI) point of view, i.e., enabling even
non-technical users to manipulate data and functionality of things in a simple way,
as we have shown in this paper.
The AAL domain presents several HCI challenges, including system accepta-
bility by the elderly, who often perceive an AAL system as a threat to both domes-
tic privacy and her/his freedom and independence. This is a psychological factor
that varies widely, as also high is the variability of the home physical space, the
technology available there or that can be installed, and the specific needs of elder-
ly. We believe that an approach based on a combination of EUD practices and IoT
technology will maximize the appropriation of AAL technology by the elderly,
making it an active part not only in the use but also in the creation and customiza-
tion of useful and usable products (services and smart objects).
References
1. Ardito C., Costabile M. F., Desolda G., Lanzilotti R., Matera M., Piccinno A., and
Picozzi M. 2014. User-Driven Visual Composition of Service-Based Interactive
Spaces. Journal of Visual Languages & Computing, 25, 4 (2014), 278-96.
2. Ardito C., Costabile M. F., Desolda G., Lanzilotti R., Matera M., and Picozzi M. 2014.
Visual Composition of Data Sources by End-Users. In Proc. International Conference
on Advanced Visual Interfaces (AVI '14). ACM, New York, NY, USA, 257-60.
3. Ardito C., Costabile M. F., Desolda G., Latzina M., and Matera M. 2015. Making
Mashups Actionable Through Elastic Design Principles. In: End-User Development -
Is-EUD 2015, Díaz P., Pipek V., Ardito C., Jensen C., Aedo I., andBoden A. Eds.
Lecture Notes in Computer Science. Springer Verlag, 236-41.
4. Atzori L., Iera A., and Morabito G. 2010. The Internet of Things: A survey. Computer
Networks, 54, 15 (2010), 2787-805.
5. Atzori L., Iera A., and Morabito G. 2010. The Internet of Things: A survey. The
International Journal of Computer and Computer Networks, 54, 15 (2010), 2787-805.
6. Barricelli B. R. and Valtolina S. 2015. Designing for End-User Development in the
Internet of Things. In: International Symposium on End-User Development, IS-EUD
2015, Díaz P., Pipek V., Ardito C., Jensen C., Aedo I., andBoden A. Eds. Lecture
Notes in Computer Science. Springer International Publishing, 9-24.
7. Ceri S., Daniel F., Matera M., and Facca F. M. 2007. Model-driven development of
context-aware Web applications. ACM Trans. Internet Technol., 7, 1 (2007), 2.
9
8. Clarkson J., Dong H., and Keates S. 2003. Quantifying design exclusion. In: Inclusive
Design, Clarkson J., Keates S., Coleman R., andLebbon C. Eds. Springer 422-36.
9. Clegg A., Young J., Iliffe S., Rikkert M. O., and Rockwood K. 2013. Frailty in elderly
people. The Lancet, 381, 9868 (2013), 752-62.
10. Coronado M. and Iglesias C. A. 2016. Task Automation Services: Automation for the
Masses. IEEE Internet Computing, 20, 1 (2016), 52-8.
11. Daniel F., Matera M., and Pozzi G. 2008. Managing runtime adaptivity through active
rules: the Bellerofonte framework. J. Web Eng., 7, 3 (2008), 179-99.
12. Desolda G., Ardito C., and Matera M. 2016. EFESTO: A Platform for the End-User
Development of Interactive Workspaces for Data Exploration. In: Rapid Mashup
Development Tools - ICWE '15, Daniel F. and Pautasso C. Eds. Communications in
Computer and Information Science. Springer International Publishing, 63-81.
13. Desolda G., Ardito C., and Matera M. 2017. Empowering end users to customize their
smart environments: model, composition paradigms and domain-specific tools. ACM
Transactions on Computer-Human Interaction, 24, 2 (2017), 52 pages.
14. AtomGraph. https://github.com/AtomGraph/Web-Client. Last access: February 12,
2018
15. Node-RED. http://nodered.org/. Last access: Sept 10, 2017
16. Lucci G. and Paternò F. 2015. Analysing How Users Prefer to Model Contextual
Event-Action Behaviours in Their Smartphones. In: International Symposium on End-
User Development - IS-EUD 2015, Díaz P., Pipek V., Ardito C., Jensen C., Aedo I.,
andBoden A. Eds. Lecture Notes in Computer Science. Springer International
Publishing, 186-91.
17. MIUR - Art. 185 ( ex 169) del Trattato UE - AMBIENT ASSISTED LIVING (AAL).
http://hubmiur.pubblica.istruzione.it/web/ricerca/ricerca-internazionale/art169/aal.
Last access: Jan 8, 2016
18. Namoun A., Nestler T., and De Angeli A. 2010. Service Composition for Non-
programmers: Prospects, Problems, and Design Recommendations. In Proc. IEEE
European Conference on Web Services (ECOWS '10). IEEE Computer Society,
Washington, DC, USA, 123-30.
19. Pane J. F., Ratanamahatana C. A., and Myers B. A. 2001. Studying the language and
structure in non-programmers' solutions to programming problems. International
Journal of Human-Computer Studies, 54, 2 (2001), 237-64.
20. Rode J. A., Toye E. F., and Blackwell A. F. 2004. The fuzzy felt
ethnography—understanding the programming patterns of domestic appliances.
Personal Ubiquitous Comput., 8, 3-4 (2004), 161-76.
21. Sixsmith A. 2013. Technology and the Challenge of Aging. In: Technologies for
Active Aging, Sixsmith A. and Gutman G. Eds. 9. Springer US, 7-25.
22. Vines J., Pritchard G., Wright P., Olivier P., and Brittain K. 2015. An Age-Old
Problem: Examining the Discourses of Ageing in HCI and Strategies for Future
Research. ACM Trans. Comput.-Hum. Interact., 22, 1 (2015), 1-27.
23. Wilkinson C. R. and De Angeli A. 2014. Applying user centred and participatory
design approaches to commercial product development. Design Studies, 35, 6 (2014),
614-31.
24. Zang N. and Rosson M. B. 2008. What's in a mashup? And why? Studying the
perceptions of web-active end users. In Proc. IEEE Symposium on Visual Languages
and Human-Centric Computing (VLHCC '08). IEEE Computer Society, Washington,
DC, USA, 31-8.
25. Zapier. https://zapier.com/. Last access: May 9, 2017
