Exploring End User Programming Needs in Home Automation

Abstract

Home automation faces the challenge of providing ubiquitous, unobtrusive services while empowering users with approachable configuration interfaces. These interfaces need to provide sufficient expressiveness to support complex automation, and notations need to be devised that enable less tech-savvy users to express such scenarios. Rule-based and process-oriented paradigms have emerged as opposing ends of the spectrum; however, their underlying concepts have not been studied comparatively. We report on a contextual inquiry study in which we collected qualitative data from 18 participants in 12 households on the current potential and acceptance of home automation, as well as explored the respective benefits and drawbacks of these two notation paradigms for end users. Results show that rule-based notations are sufficient for simple automation tasks but not flexible enough for more complex use cases. The resulting insights can inform the design of interfaces for smart homes to enable usable real-world home automation for end users.

Full text

11
Exploring End User Programming Needs in Home Automation
JULIA BRICH, MARCEL WALCH, MICHAEL RIETZLER, and MICHAEL WEBER,
Ulm University
FLORIAN SCHAUB, Ulm University, Carnegie Mellon University, and University of Michigan
Home automation faces the challenge of providing ubiquitous, unobtrusive services while empowering users
with approachable configuration interfaces. These interfaces need to provide sufficient expressiveness to
support complex automation, and notations need to be devised that enable less tech-savvy users to express
such scenarios. Rule-based and process-oriented paradigms have emerged as opposing ends of the spectrum;
however, their underlying concepts have not been studied comparatively. We report on a contextual inquiry
study in which we collected qualitative data from 18 participants in 12 households on the current potential
and acceptance of home automation, as well as explored the respective benefits and drawbacks of these two
notation paradigms for end users. Results show that rule-based notations are sufficient for simple automation
tasks but not flexible enough for more complex use cases. The resulting insights can inform the design of
interfaces for smart homes to enable usable real-world home automation for end users.
CCS Concepts: rHuman-centered computing →Empirical studies in interaction design;Ubiqui-
tous and mobile computing systems and tools;rSoftware and its engineering →Software notations and
tools;
Additional Key Words and Phrases: Configuration interfaces, contextual inquiry, qualitative analysis, smart
home
ACM Reference Format:
Julia Brich, Marcel Walch, Michael Rietzler, Michael Weber, and Florian Schaub. 2017. Exploring end user
programming needs in home automation. ACM Trans. Comput.-Hum. Interact. 24, 2, Article 11 (April 2017),
35 pages.
DOI: http://dx.doi.org/10.1145/3057858
1. INTRODUCTION
Home automation as a concept has been around since the 1960s, while the term “smart
house” was coined in 1984 [Aldrich 2003]. The purpose of home automation is to support
inhabitants in their daily lives via technological means. Areas of interest include secu-
rity functions like access control and surveillance systems, resource management with
regard to water and electricity, multimedia functions for ubiquitous content stream-
ing, and comfort functions like light and heat management [Bartram et al. 2011; Ur
et al. 2013; Takayama et al. 2012; Yang and Newman 2013]. The profiles of smart
home users are naturally diverse. With regard to technical interest and investment,
the spectrum ranges from highly involved Do-It-Yourselfers to pure consumers of the
Plug‘n’Play mentality. Related to this, solutions for various budget sizes have been
developed. Since the state of a building is an impacting factor for home automation,
Authors’ addresses: J. Brich, M. Walch, M. Rietzler, and M. Weber, Institute of Media Informatics, Ulm Uni-
versity, Germany; emails: {julia.brich, marcel.walch, michael.rietzler, michael.weber}@uni-ulm.de; F. Schaub,
School of Information, University of Michigan, USA; email: fschaub@umich.edu.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted
without fee provided that copies are not made or distributed for profit or commercial advantage and that
copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for
components of this work owned by others than ACM must be honored. Abstracting with credit is permitted.
To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this
work in other works requires prior specific permission and/or a fee. Permissions may be requested from
Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212)
869-0481, or permissions@acm.org.
c
2017 ACM 1073-0516/2017/04-ART11 $15.00
DOI: http://dx.doi.org/10.1145/3057858
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:2 J.Brichetal.
home automation systems exist in various degrees of building integration, ranging from
fixed home installations for new buildings to lightweight wireless solutions that can be
added without modifying the actual infrastructure. With regard to age and ability of
inhabitants, the specialized segment of Assisted Living [Chan et al. 2008; Kleinberger
et al. 2007] deals with the additional requirements of senior users or in-home patients.
This diverse user profile makes interaction design and development for smart homes
challenging. On the one hand, solutions should be ubiquitous and refrain from impact-
ing the user’s life; on the other hand, people expect a certain degree of control over
what is going on in their own homes [Bernheim Brush et al. 2011; Costanza et al. 2014;
Mennicken and Huang 2012a; Mennicken et al. 2014; Rodden et al. 2013]. A number
of commercial home interfaces have thus opted to expose the underlying technology in
staggering detail to enable users full access to the home’s functions [HomeSeer 2016;
Insteon 2016; Loxone 2016; myGEKKO 2016]. This however conflicts with the reality
that only part of the user population is both tech-savvy and interested enough to pro-
gram their home from the ground up [Bartram et al. 2011; Bernheim Brush et al. 2011;
Poole et al. 2008; Rodden et al. 2013; Rode et al. 2005; Yang and Newman 2013]. Home
automation interfaces thus need to provide control solutions that also cater to the re-
quirements of less tech-savvy users that still want to enjoy the benefits of an automated
home. Current attempts for simplified interfaces usually provide home configuration in
the form of if–then rule-based approaches [De Russis and Corno 2015; Dey et al. 2006;
IFTTT 2016; Ur et al. 2014; Zamora-Izquierdo et al. 2010]. Other works have tried
process-oriented approaches in the hope of providing increased expressiveness while
still retaining easy-to-comprehend visualizations [Dahl and Svendsen 2011; Rietzler
et al. 2013; Walch et al. 2013]. In their strictest form, these two paradigms can be seen
as opposing ends of the home programming spectrum, and provide cornerstones for in-
teraction research focused on end user needs. The fundamental principles underlying
these two paradigms have however not yet been studied with regard to their effects
on expressiveness and usability. Also, it is yet unclear how far-reaching end users’
automation needs truly are. Research focused on the benefits and drawbacks of these
basic paradigms can help to identify useful features and strategies for usable home
automation interfaces targeting end users.
We conducted a contextual inquiry study with 18 participants in 12 households
in order to gain a better understanding of end user needs for home automation and
respective configuration. Through a process consisting of multiple home visitations and
semi-structured interviews, we encouraged participants to think of and model home
automation use cases in the context of their own home and daily activities. In a between-
subjects design, participants either used a rule-based or a process-oriented notation to
record automation tasks in order to study practical expressiveness, complexity, and ease
of use of the two approaches. We opted for a pen and paper study to encourage creativity
and the identification of automation use cases relevant to their own daily routines and
household devices, without artificially restricting the elicitation process to a specific
user interface implementation of either notation. Additionally, we also investigated
general notions and perceptions on home automation in order to identify real-world
deployment issues and requirements. Our results show that rule-based notations are
generally sufficient to express simple automation tasks but are not flexible enough to
support more complex use cases. Participants that used the process-oriented notation
valued its expressiveness and presented us with more complex use cases involving
more devices in their homes. We further discuss how our results can inform the design
of end user configuration interfaces for smart home technology in order to enable usable
real-world home automation for end users.
The remainder of this article is structured as follows. First, we discuss related
work dealing with the acceptance and requirements of home automation by end users.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:3
Furthermore, we detail various projects that have developed configuration interfaces
following the rule-based and process-oriented paradigms. From this analysis, we de-
rive the research questions that were addressed in our study. We then outline the
particulars of our study and present the resulting insights.
2. RELATED WORK
Several previous studies aimed to elicit relevant factors for the adoption of smart home
technology. We first discuss their results before focusing on proposed rule-based and
process-oriented configuration approaches.
2.1. Requirements for Successful Home Automation
As the adoption of home automation in the general populace has been reluctant, re-
search has been conducted over the years to better understand the requirements of
potential users.
2.1.1. Notable Characteristics of Home Automation.
To provide designers with an easy and
reusable means of grouping home automation solutions targeting certain routines,
Crabtree et al. [2002] expand on Alexander’s approach of architectural design patterns
inspired by social behavior [Alexander et al. 1977]. They designed a pattern-based
approach to structure the domestic behavior of home automation study participants
and introduced the notion of activity centers as the basis of domestic routines involving
technology. Zhang et al. [2009], in their work on home automation interface complexity,
identified home automation tasks as belonging to one of three categories, namely skill-
based (e.g., turning on the lights), rule-based (e.g., If it is dark outside, close the blinds),
and knowledge-based tasks. The latter comprises a more complex network of rule-based
tasks and requires choices to be made depending on more than one context condition.
With regard to the static nature of task automation, several works have pointed out how
home automation will have to deal with irregular behavior or user routines changing
over time [Davidoff et al. 2006; Chan et al. 2008; Kaasinen et al. 2012; Costanza et al.
2014].
2.1.2. Diverse Target Population.
Davidoff et al. [2006] put their focus on researching
the requirements of families with regard to home automation. They found flexibility
of the system to be paramount as participants expressed a need for their system to
accommodate their ever-changing routines. According to Davidoff et al., automation
systems should provide users with capabilities to easily create and modify automation
behavior. The survey conducted by Kaasinen et al. [2012] concerns itself with the gen-
eral notion of user acceptance and expectations in intelligent environments. Among
other things, they point out the importance of ease of use and giving the user a sense of
being in control. They define the user as an active co-crafter in intelligent environments
rather than a passive consumer and advise designers of intelligent environments to
create technology that fits into their users’ routines. Furthermore, smart technology
should be designed for all kinds of users and neither exclusively for experts nor novices.
With similar results, Mennicken and Huang [2012a] conducted an in-the-wild study
that followed home automation professionals and end users in the process of estab-
lishing and inhabiting smart homes. Their findings include the notion of supporting
and involving end users through all phases, from smart home conception to daily use.
Regarding the latter, they also point out that capabilities need to be present to support
both users with a high interest in modifying the home on a technical level as well
as more passive users of the infrastructure. Pierce and Paulos [2012] reviewed HCI
literature with a focus on energy-related home automation approaches. They found
that upcoming smart grid technology will have a huge impact on home users who
previously had little contact with energy technology due to the centralized nature of
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:4 J.Brichetal.
current provider systems. Aside from the persuasive aspect of getting users to reduce
or shift energy consumption commonly associated with smart energy technology, they
found that interaction research also needs to account for cultural aspects surrounding
energy usage in the home. Special attention will have to be payed to the reconcili-
ation of users’ personal energy needs and routines with requirements characteristic
of smart grids, such as peak avoidance. Further suggestions point toward automat-
ing everyday appliances like thermostats and dishwashers—a notion that has since
been taken up by the industry. Pierce and Paulos further encourage research on new
interactions to empower end users and leading them into their new role of energy sup-
plier and distributor facilitated by an increasingly flexible energy grid. Costanza et al.
[2014] conducted an in-the-wild study to research the impact and reception of an agent-
based system designed to help users organize their laundry sessions in the context of
smart grid technology. They found that doing laundry is an especially delicate use
case illustrating the area of potential conflict between user routines and automation
technology as users have rituals as well as necessities surrounding the laundry task
that cannot easily be shifted to accommodate, for example, the goals of energy-aware
appliances.
2.1.3. Barriers to Adoption.
Harper [2003] identified cost and outdated building struc-
tures as potential impediments. Chan et al. [2008] point toward the large variety of
connection technology and protocols. They also address legal and ethical issues rel-
evant for the eHealth sector. Newman et al. [2008] identify “piecemeal interaction”
as a central problem, i.e., that functionality is often scattered across various devices,
services, and interfaces with narrowly defined responsibilities. Eckl and MacWilliams
[2009] have found smart homes to be either too complex to use (stemming from the
intention to be a general purpose system), too restricted in their functions (special-
ized niche products) or too “clever” by trying to take the user out of the control loop.
They also note cost as a relevant factor. Brush et al. [2011] conducted an in-the-wild
study with DIY households and more consumer-oriented households that had out-
sourced their home automation technology to specifically identify potential barriers
for the widespread adoption of home automation technology. They identified four bar-
riers, namely high cost of ownership, inflexibility, poor manageability, and difficulty
achieving security. Poor manageability in particular addresses how complex user in-
terfaces can confuse and irritate users. While their participants successfully incor-
porated home automation into their lives, they had to overcome diverse challenges
that might keep less enthusiastic end users entirely from adopting home automation
technology.
2.1.4. Interface Considerations.
Several studies have found that full automation is not
necessarily in the interest of users [Koskela et al. 2004; Koskela and V¨
a¨
an¨
anen-Vainio-
Mattila 2004; Hwang and Hoey 2012; Leitner et al. 2013], because they may fear losing
control over the technology. Instead of full and invisible automation, end user configu-
ration of automation behavior is proposed. Harper [2003] however found the usability of
interfaces lacking and criticized the missing involvement of users in the design process.
In the context of Assisted Living, Chan et al. [2008] also advocate user involvement
at the design stage. Zhang et al. [2009] studied the effect of interfaces varying with
regard to presented complexity on the completion metrics of home automation tasks
and found that skill-based tasks as described above can be accomplished more effec-
tively with a less complex, direct control style interface, while rule-based tasks are
better supported by a more complex, task-oriented interface. They recommend the use
of both approaches in home automation interfaces to adequately support the variety
of tasks. Mennicken et al.’s [2014] survey compiles further challenges from related
work. They identify the need to provide users with the ability to model what they find
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:5
useful instead of what is technically feasible or interesting from the researcher’s point
of view. However, information needs to be visualized in a reduced way so even less
tech-savvy users can make sense of it. They further point out the opportunities that
could arise from users sharing their automation ideas and behaviors. They emphasize
the importance of studying home automation interaction in the wild to uncover user
requirements in more detail. In relation to this, Costanza et al. [2014] point out that
interaction touching upon user routines needs to be designed in a way that allows
users to incorporate the technology into their everyday lives instead of building their
life around the automation system.
2.1.5. Device Integration.
Another issue for the successful adoption of home automation
is centered around the issue of first bringing smart devices into the home. While this
has been pointed out by research for some time [Poole et al. 2008; Mennicken and
Huang 2012a; Chong et al. 2014] and various approaches have been proposed, it has
not been fully addressed. Recently, Jewell et al. [2015] conducted usability studies on
how to design configuration interfaces for the integration of smart devices aimed at less
tech-savvy users. They found that end users require the interfaces to be unobtrusive,
self-explanatory, and primarily robust.
Summarily, the following areas can be identified from the related work as critical
for the successful adoption of home automation by end users. External factors related
to the infrastructure deal with the potentially high cost and effort to turn current
buildings into smart homes. In conjunction with this, surrounding infrastructure like
high-speed internet and smart grid access need to be up to par. Furthermore, current
smart home solutions are often niche products advertised for assisted living, entertain-
ment, or security rather than holistically integrated systems. This also leads to issues
with installing and managing home automation systems that serve multiple purposes
as their combination tends to be complex and comprised of heterogeneous technologies.
Setting up a smart home requires expertise that either needs to come from an external
contractor or be acquired by end users themselves. In this regard, it needs to be kept
in mind that not all end users are interested in managing the technology; often they
“just want it to work.” Systems should thus make sure to appeal to users with different
levels of expertise. Also relating to technology is the issue that most current systems
fail to embody multi-user settings that regularly occur in reality. And finally, for in-
ternal factors like acceptance and trust, home automation systems have to be reliable,
inform users about their inner workings, and be flexible enough to be integrated into
users’ routines and everyday lives. The quality of user interfaces is paramount for the
adoption of highly complex, distributed systems like home automation—users need to
be able to monitor and interact with the automation system; however, the available
information needs to be condensed and presented in a usable fashion.
In our study, we approach the issue of requirements for successful home automation
from the user’s perspective. We are interested in how prospective end users themselves
evaluate the potential of home automation for their personal lives and what they per-
ceive as inhibiting or facilitating factors of the technology. This can help to either verify
and strengthen the points addressed in the related work and help identify additional
areas of concern. Based on our results, both researchers and industry can gain a bet-
ter understanding of how to take home automation from interesting novelty to broad
adoption in the general population.
In addition, we found it beneficial to study more closely the design of current smart
home configuration interfaces as the related work presented so far has highlighted that
empirical research targeting the underlying interaction paradigms is missing. Related
work pertaining to smart home configuration interfaces in general, and rule-based and
process-oriented approaches in particular, is presented in the following.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:6 J.Brichetal.
2.2. Smart Home Configuration Interfaces
Prior work on end user configuration of home automation has uncovered interesting
aspects with regard to programming metaphors and user requirements. Rode et al.
[2004] define device programming as an extension of direct manipulation. They iden-
tified “ahead of time” (i.e., scheduling the execution of an activity) and “repeats easy”
(i.e., the automation of recurring tasks) as main programming functions. They also find
that users only choose programming over direct manipulation if there is a powerful
necessity for it. Truong et al. [2004] studied how users conceptualize automation ap-
plications without a pre-given model. Their participants tended to specify desires by
abstracting from devices and adopt a more task-centric perspective. They presented
CAMP, a configuration interface introducing the fridge magnet metaphor. Newman
[2006] and Newman et al. [2008] collected information on the adequacy of classic pro-
gramming constructs for use by non-tech-savvy users and came up with the notion of
digital recipes for their OSCAR interface, which encapsulate scenes in which users may
specify device settings to be executed in combination. Koskela et al. [2004], Koskela
and V¨
a¨
an¨
anen-Vainio-Mattila [2004], and Leitner et al. [2013] also focused on scene-
based end user configuration. Both approaches initially focused on time triggers for
the execution of scenes, Koskela et al.’s participants however suggested the addition
of context event-based triggers. They further uncovered two distinct activity patterns
desired by users: pattern control and instant control. Pattern control denotes cases
where users create automation settings, whereas instant control refers to remote con-
trol options. Participants preferred to use computers for pattern control and mobile
phones for instant control. Eckl and MacWilliams [2009] also propose a system where
users can invoke scenes; however, the respectively available selection of scenes depends
on the current context. They advocate that the user should make a conscious choice
instead of invisible background automation. Both Brush et al. [2011] and Leitner et al.
[2013] find existing commercial home automation interfaces lacking in usability. Even
computer science (CS) students had difficulties mastering the evaluated programming
interfaces [Leitner et al. 2013].
In summary, the related work has identified that automation interaction can con-
stitute either direct manipulation/remote control functionality, or occur in the form of
programming. Both are demanded by users, the more complex programming variant is
however only attractive if it offers clear additional value over the remote control alter-
native. With regard to abstraction, users tend to be task-oriented rather than focusing
on the specific smart device. Scenes constitute a common automation construct that
has also been adapted in the industry; thereby, a combination of related settings can
be triggered at a given time or via determined context. This is however a very basic
form of automation; its characteristics are closer to remote control functionality than
to home programming. We will focus instead on the latter to gain a better understand-
ing of the human computer interaction required to create more complex automation
scenarios. With regard to user involvement, related work from this paragraph and the
section before both find that fully hidden background automation can be detrimental
to user acceptance and trust. Instead, end users want and need to be involved with
the automatic processes happening in their homes. For users to be involved in the
configuration of their smart homes, however, interfaces need to be easy to understand
and usable even for less tech-savvy users.
2.2.1. Rule-Based Configuration.
Many recently developed end user configuration inter-
faces of home automation systems focus primarily on rule-based approaches follow-
ing the if–then paradigm (also referred to in the related work as trigger-action or
event-action paradigm). The Stick-e Note architecture [Pascoe 1997] early on com-
bined context conditions as annotations with desired actions “written” on Post-It notes
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:7
to raise awareness for context-aware computing. The iCAP system [Dey et al. 2006]
provides a visual programming interface for context-aware applications, which allows
the specification of if–then rules that are triggered by certain context configurations.
An iCAP rule may combine up to three conditions to trigger one action. The interface
of the DOMOSEC prototype home [Zamora-Izquierdo et al. 2010], among other things,
enabled users to configure scenarios in which a number of device actions could be ex-
ecuted under certain conditions. While those scenarios were not termed as rules, they
exhibit if–then semantics. Garc´
ıa-Herranz et al. [2010] developed a system-level rule
engine to provide an application-independent mechanism for end user specification of
rules. They intended to ameliorate the trade-off between flexibility and simplicity for
users, with the vision of generating situation-specific interfaces for the underlying rule
system. In addition to the if–then rules, they introduced triggers (a when condition)
and timers for time-dependent functions. This can be considered a hybrid approach
combining aspects from both conventional rule-based and process-oriented notations
as introduced in the next section. The HomeMaestro application [Salzberg 2011] for
Microsoft’s HomeOS [Dixon et al. 2012] extends the explicit programming concept by
enabling users to record rules from interaction within their actual environment. Ur
et al. [2014] examined the capabilities of rule-based programming for end user config-
uration in the smart home context. They asked participants to freely submit desired
smart home behaviors and extracted those that could be expressed as rules via one
trigger–one action or multiple triggers–multiple actions patterns. They found end user
configuration to be feasible as well as necessary due to the high variety in desired
behaviors. A recent analysis of 200,000 rules defined by users of the web service IFTT
[2016] shows the real-world proliferation and acceptance of rule-based programming
[Ur et al. 2016]. De Russis and Corno [2015] developed HomeRules to test a set of
guidelines for rule-based programming derived from related work. HomeRules is a
tangible interface similar to HomeMaestro that combines real-world device interaction
with rule-based programming. They evaluated the concept in a qualitative study and
found participants both capable of and enthusiastic about rule-based programming.
To summarize, rules are constructed ofconditions comprised from context and result-
ing automation actions, following the structure of “if THIS, then THAT.” In addition
to this basic version, several variations have been proposed where the number and
combination of conditions can vary, where an additional “when” construct has been
introduced or where actions can be triggered by a timer. As large rule systems can
become complex and difficult to oversee, some projects have conceived workarounds
like context filters or recording functionality. All in all, rule-based configuration is the
most widespread home automation programming paradigm in research and industry
and has been well-received by users. However, as rules follow a strict pattern and are
not naturally adequate for automation chains, another paradigm has evolved—process-
oriented configuration.
2.2.2. Process-Oriented Configuration.
Process-oriented notations have emerged for the
expression of temporal and loosely time-synchronized activities. However, the expres-
siveness of process-oriented notations is often coupled with complex user interfaces.
Although some of the above mentioned rule-based solutions support sequences of ac-
tions, it is not an inherent characteristic of rule-based notations as defined by Ur et al.
[2014] and employed in many current home automation systems where one or more
trigger events are directly responsible for the execution of one or more actions.
Humble et al. [2003] propose a more process-oriented approach based on a jig-saw
metaphor to enable users to create multi-stage connections of sensors and devices
as an information flow pipeline. In accordance with the source-to-sink control flow
paradigm, this introduces a timing aspect as a device’s action and subsequent state
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:8 J.Brichetal.
change trigger the next action in the process. Dahl and Svendsen [2011] investigated
the jig-saw metaphor, filtered lists, and wiring diagrams to elicit usability factors for
end user configuration. Filtered lists constitute a basic rule-based interface where
users specify a trigger and resulting action in strict order. Wiring diagrams allow
for the placement and connection of component blocks via directed arrows and the
subsequent specification of the desired behavior. While they find that filtered lists
were rated as most effective, participants found the more process-oriented notations
to be more enjoyable. Rietzler et al. [2013] and Walch et al. [2013] propose a process-
oriented graphical end user configuration approach for smart homes, similar to wiring
diagrams, which reduces the complexity of process-oriented notations while retaining
their expressiveness. However, the work reports only preliminary usability results of
this approach based on a prototype implementation of the homeBLOX system.
In summary, processes-oriented configuration can be seen as the creation of multi-
stage automation chains where one set of conditions and actions can trigger one or more
following sets. Process-oriented configuration introduces a temporal aspect by nature
and allows conditional branching. While some interesting approaches for the paradigm
have been proposed, it has not been thoroughly explored from an interaction research
perspective. It is yet unclear whether the benefits of process-oriented configuration
warrant a more complex user interface from an end user’s point of view.
2.2.3. Discussion of Configuration Approaches.
It can be concluded that existing home
automation configuration approaches have a number of drawbacks. Scene and sched-
ule programming [Koskela et al. 2004; Koskela and V¨
a¨
an¨
anen-Vainio-Mattila 2004;
Leitner et al. 2013; Newman et al. 2008; Rode et al. 2004], which is used by the
commercial Control4Home [2016] and e-Domotica [2016] systems, is limited in ex-
pressiveness. Existing interfaces also tend to suffer from Newman et al.’s “piecemeal
interaction,” e.g., users have to control settings for lights, thermostats, etc., separately.
Rule-based interfaces further tend to be rather programming-oriented (e.g., Leitner
et al. [2013] or Loxone [2016]), which may introduce barriers for less tech-savvy users.
More visual systems like Ninja Blocks [2016] and If-This-Then-That [IFTTT 2016] lose
clarity when an automation task requires multiple rules. Rule-based approaches have
also been found to limit the user’s overall creativity and expressiveness [Dahl and
Svendsen 2011; Garc´
ıa-Herranz et al. 2010], whereas process-oriented notations need
to focus more on supporting clarity of control flow mechanics [Dahl and Svendsen 2011].
With regard to programming notations rather than scene setting, both rule-based and
process-oriented configuration seem to have the potential to be employed for effective
end user programming. The respective advantages and drawbacks of the two paradigms
have however not formally been researched. Doing so can lead to a sound understanding
of end user interaction requirements and inform the design of future home automation
interfaces. The next section gives a summary of our rationale and details the research
questions we derived from our analysis of related work.
3. RESEARCH QUESTIONS
While home automation has been extensively researched and many home automation
systems and configuration interfaces have been proposed, the concept has not yet pen-
etrated the consumer market on a larger scale [Bernheim Brush et al. 2011; Kaasinen
et al. 2012; Newman et al. 2008]. Current solutions are marked by high costs, high com-
plexity, low flexibility, and low usability [Bernheim Brush et al. 2011; Harper 2003]. To
further the adoption of smart home technology by end users, home automation systems
need to offer intuitive, easy-to-use configuration capabilities and interfaces that enable
users to tailor their home’s behavior to their individual needs. However, home automa-
tion interfaces are faced with the challenge of presenting highly technical automation
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:9
procedures in a way that is intelligible to non-expert users, yet being expressive enough
to support their automation needs.
Both in industry and research, primarily if-this-then-that rules are used to specify
home automation behavior: Based on time- or event-based triggers, certain automa-
tion actions are performed, e.g., “when the alarm clock rings, turn on the lights.” While
rule-based approaches allow for accessible automation of simple tasks, more complex
automation scenarios—that need to assume the form of extensive rule systems—are
both difficult to create and potentially difficult to understand or recognize upon re-
visitation. Recent research has proposed to employ a more process-oriented approach
to smart home end user configuration [Dahl and Svendsen 2011; Rietzler et al. 2013;
Walch et al. 2013] in order to enhance the expressiveness of automation tasks.
To unearth the inherent benefits and drawbacks characteristic of the rule-based
and process-oriented configuration paradigms, we designed a contextual inquiry study
in which participants worked with notations designed to be representative of each
of the paradigms for one week in a natural setting surrounded by their own devices
and everyday routines. We felt that this would yield more meaningful results than
evaluating the notations in a lab setting with artificial use cases that were of no
personal relevance to participants. Also, we hypothesized that we could gain more in-
depth feedback if we let participants engage with the notation for a longer period of
time. In addition, we also investigated the general attitude of our participants toward
home automation to identify real-world deployment issues and requirements for the
successful adoption of home automation by the general population. The underlying
research questions of our study are as follows.
Potential of Home Automation: Both as a survey of the current technological state of
households and to get participants to start imagining home automation in their own
life, we were interested in recording device inventories. In these inventories, we col-
lected information on both appliances and electronic devices that participants already
owned or that they desired to have. We further noted whether participants owned
devices that were already network-enabled to estimate how much and what kind of
home automation could potentially be established in current households. We used the
compilation of these inventories further to get participants to think about use cases for
the devices they mentioned in combination with their daily routines. The aim of this
was to potentially reveal common use case clusters that should be the target of home
automation industry efforts in the future.
Acceptance of Home Automation: To be able to put our results into perspective, we
inquired about participants’ general feelings toward home automation, as we expected
a variety of attitudes, ranging from highly enthusiastic to very reserved. Related to
this notion, we wanted to determine the conceptual and technological prerequisites
necessary for end users to embrace home automation.
Rule-Based vs. Process-Oriented Notation: As the main focus of our study, we pre-
sented participants with either a rule-based notation for home automation or a process-
oriented one to determine which features and strategies of each can be employed to
create usable home automation interfaces. We expected rule-based notations to be suf-
ficient for simple automation scenarios but were interested in whether participants
would feel limited by it. Vice versa, we expected that process-oriented notations were
more difficult to grasp but yield a higher expressiveness. Related to this, it was our goal
to determine whether users even feel the need for complex home automation scenarios.
4. STUDY PROCEDURE
To address the research questions, we devised the following study design. We visited
participants in their home to conduct a home tour and spark their creativity toward
home automation by letting them formulate use cases in an unrestricted manner,
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:10 J.Brichetal.
Fig. 1. Distribution of the research tasks between the two study conditions.
including those devices, appliances, and objects identified during the home tour as
well as any other kind of sensor, device, or actuator they could imagine. Afterwards, we
exposed them to a pen and paper notation kit following either the rule-based or process-
oriented paradigm. Participants were instructed to try out the notation with one think
aloud use case, while the experimenters were present to help with misunderstandings
and getting participants acquainted with the respective notation. We would then leave
participants alone with the notation for one week to allow them to think about and
create desired automation procedures as they noticed them in their daily lives. In this,
they were not restricted to the devices or sensors they owned; we encouraged them
to model whatever behavior they would want their ideal smart home to have. During
a subsequent visit, participants were encouraged to present their devised use cases
followed by a semi-structured interview. The interview was basically the same for both
conditions aside from the terminology of rules and processes. Condition-specific were
those questions that dealt with the particulars of each notation as detailed below.
To conclude, participants were given a recognition task. This task comprised three
interpretation exercises, i.e., participants were given one unfamiliar use case recorded
in the notation that was previously introduced to them, one from the respective other
study condition and one use case recorded in an adapted process notation for additional
results. Participants were then instructed to tell the experimenter what home behavior
they thought would result from each of the automation procedures. This task was added
to examine the understandability of the notations. The distribution of the research
tasks between the two study conditions can be seen in Figure 1.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:11
In the following, we first present the definitions of rule-based and process-oriented
notations that were employed in this study. Furthermore, we introduce the methods
and materials that were used during the study to gather information from partici-
pants relating to the research questions formulated above. To conclude, we present the
recruitment process and participant demographics.
4.1. Definition of Notations
Our goal was to gain a deeper understanding of the characteristic benefits and draw-
backs of rule-based and process-oriented notations in relation to end user configuration
of smart homes. For this purpose, we first derived the relevant characteristics of each
notation approach based on our analysis of existing research and commercial inter-
faces. We then proceeded to define generalized notations being representative for the
respective approach. This was done to be able to focus on the underlying principles
distinctive of each paradigm and to avoid implementation-specific issues that would
have been introduced by comparing existing notations. While for example rule-based
notations can be extended with “when” constructs or chaining of rules to serve similar
purposes as process-oriented notations, we wanted to explore the fundamental concep-
tual differences between the paradigms and thus chose to present participants with a
strict version of rule-based notations in our study.
4.1.1. Rule-Based Notation.
In strict rule-based notations, automation behavior is de-
finedinanif THIS, then THAT form. The if-clause may contain points of time (e.g.,
8a.m.), device events (e.g., bedroom lights on), or contextual events (e.g., Alice entered
living room). The then-clause describes device actions (e.g., turn on the radio)tobe
executed after successful evaluation of the if-clause. The vocabulary for the elements
of rule notations varies. We defined the if-clause as a trigger and the then-clause as
resulting actions, similar to Ur et al. [2014]. Commonly, triggers can be constructed by
combining several events via the use of AND-operators. OR-constructs are represented
by separate rules with a complementary set of trigger events. Actions can be combined
with AND-constructs, i.e., a trigger results in the execution of multiple actions. We did
not include additional timing elements, like those suggested by Garc´
ıa-Herranz et al.
[2010], as they constitute specific extensions not found in most rule-based configura-
tion interfaces for home automation. Similarly, we did not include chaining of rules
(e.g., if THIS, then RULE X), because it constitutes a timing-dependent pre-condition
for rule x—a feature found in some rule-based notations, but not common in existing
configuration interfaces and thus not representative of the underlying paradigm.
An example of our generalized rule-based notation can be seen in Figure 2. In the
upper left corner, a name for the respective rule has to be specified. Triggers and actions
can then be specified in dedicated columns. The AND semantic is used implicitly on both
sides, i.e., if more than one trigger is specified, then all triggers need to evaluate true
to trigger the action column; similarly, all actions in the action column are activated in
parallel. For the OR semantic, a new rule has to be created. Both triggers and actions
are device-specific and need to be specified in the format device (event) or device (action).
Sensors, which are essential for smart environments, are not necessarily treated as
dedicated devices in our notation; instead, they have been abstracted into the concept
of context. This adheres to the task-centric perspective of end users as postulated
by Truong et al. [2004] and is deemed to be easier to understand for non-tech-savvy
users. Context events are to be assigned either to their respective sensing devices
as understood by the user like clock (between 6 and 8 am) for time or thermometer
(>32◦C) for temperature or they can be labeled with their associated variable, i.e.,
humidity (high). More advanced sensing is supported but is dependent on the respective
technical knowledge of the user. In general, participants are free to come up with any
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:12 J.Brichetal.
Fig. 2. Example of rule-based notation.
kind of context they can imagine—regardless of technical understanding or feasibility.
Additionally, we included a timer construct where events in rule A could trigger the
virtual device timer (set to x minutes) and timer (elapsed) could in turn trigger the
actions in rule B. This is a feature used in most current rule-based approaches and
does not break the integral “if THIS, then THAT” structure of the paradigm.
4.1.2. Process-Oriented Notation.
Process-oriented notations allow for the specification
of control graphs. Devices and context elements can be depicted as nodes, while edges
denote control flow. For every connection, triggering event and resulting action have to
be specified. Logical operators (AND and OR) can be used between the nodes to create
arbitrarily complex automation processes. Since the control flow is dependent on the
triggering and completion of previous elements, time dependencies are introduced.
Thus, a chain of events and actions accrues a growing context description statement
for later elements. Consider the process “When I wake up in the morning, brew a cup
of coffee. On weekends, turn on the radio when coffee is ready, otherwise the TV news
channel. If my partner is still asleep 20 minutes later, open the bedroom blinds.” The
coffee maker action is triggered every time the user wakes up, while the blinds action
requires the respective aggregated context to be executed. It is further possible to fully
or partially encapsulate processes as reusable subprocesses, which corresponds to the
concept of rule chaining.
An example of our process-oriented notation can be seen in Figure 3. In the upper
left corner, a name for the respective process has to be specified. Every process has
a designated starting point. Arrows start at that point and connect to the first order
of devices. More devices can be added by adding more arrows, both horizontally and
vertically. Each device has events and actions, e.g., alarm clock has the action “set
alarm” and the event “ringing.” Events and actions need to be specified in the format
device (event) or device (action). Mirroring the rule-based notation, context events are to
be assigned to their respective sensing devices like clock (between 6 and 8 am) for time
or thermometer (>32◦C) for temperature. Based on the same argumentation as before,
advanced sensing is possible but depends on the user’s technical understanding. AND
and OR operators can be placed at the appropriate positions of a process. All incoming
edges of an AND operator need to be fulfilled for progression while for an OR operator
one suffices. We also included a timer device with the action “set to xminutes” and the
event “elapsed.” Processes can further be reused as subprocesses, referred to with their
name.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:13
Fig. 3. Example of process-oriented notation.
4.2. Methods and Materials
To address the research questions regarding acceptance and potential of home automa-
tion as well as interaction with the programming notations, we opted for a qualitative
study design. We created a number of supporting materials that are introduced in the
following. Originally, all materials were in German to accommodate our participants.
4.2.1. Home Tour.
To start off the study, we introduced participants to the general idea
of home automation and gave them an example to familiarize themselves with the
concept. We chose the example of “When the alarm clock rings in the morning, coffee
is automatically made.” Participants were informed that they could stop participating
at any time but that they would receive the compensation of 30 Euros only upon
completion of the entire study. They had to consent to audio recordings of the interviews
as well as to having pictures taken of their handwritten notations. Subsequently, they
were given the opportunity to ask questions before signing a consent form.
Following the formalities, participants were asked to guide the experimenters
through their home. For this, we devised a supporting questionnaire for the experi-
menter to note the participating household’s ID, what kind of appliances and electronic
devices already existed in the household or were desired and what use cases the par-
ticipants could imagine in their environments.
For the device inventory, we devised a matrix of the most common rooms (e.g., living
room, kitchen, bathroom, home office, etc.) and potentially automatable devices (e.g.,
lights, windows, stove, multimedia equipment, door locks, etc.) with extra fields for
additional rooms or devices so the experimenter could note existing devices quickly
without disrupting the natural flow of the home tour. The experimenter could also
note whether devices (1) exist, whether they (2) exist and are already network-enabled,
or whether they (3) do not exist but are desired by the participants.
As for the use cases, participants were encouraged to formulate these in a free form
manner while disregarding any technical constraints. They should state whatever be-
havior they would want from an automation system—including additional devices, sen-
sors and actuators—and not be limited by what they thought feasible. Experimenters
could point out existing devices to elicit further use cases but should not suggest any;
i.e., they could ask participants if they would find it useful to automate a particular
device x, but not tell them what automation in that case could look like so as not to
prime them towards the experimenter’s notion of home automation.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:14 J.Brichetal.
The home tour was generally conducted with two experimenters present, so one could
lead the tour while the other took notes. Exceptions were made when necessary. If the
participating household consisted of two participants, the home tour was conducted
together. During the tour, audio data was collected for later use case analysis.
4.2.2. Notation Kits.
Afterward, the participants were introduced to the notation kits
and given the opportunity to ask questions. For both notations, we devised paper-
based notation kits. Paper-based approaches are especially helpful in the early stages
of interaction research to ensure that participants can complete tasks in a way that
feels intuitive to them. Rodden et al. [2013], building upon the work of Tohidi et al.
[2006], determine the advantages of sketches to be that they are (1) disposable—which
encourages participants to criticize aspects without fear of insulting a researcher’s
work as could be the case with fully developed interfaces, (2) minimalist—which helps
both researchers and participants to focus on the core aspects of the research ques-
tions instead of having to deal with additional aspects introduced by technology, (3)
explorative—which allows participants to come up with creative solutions that are not
fenced in by what is possible or not in a specific prototype, and last (4) ambiguous—in
a way that participants can appropriate the sketched content more easily and make it
their own. It has to be noted that paper-based notations can only go so far to support
the design of future interfaces. While conclusions can be drawn concerning the appli-
cability of certain interaction metaphors, implementations of the notations will have
to prove their merits by themselves. Usability testing will then reveal whether a given
interface is indeed capable of conveying its underlying interaction paradigm to end
users in a usable way both with regard to clarity and efficiency. This will be especially
crucial once features from different paradigms are combined in the hope of capitalizing
on their varying advantages. Paper-based research is however helpful to include the
intended end user in the design process early on and then guide the initial design
choices of future implementations. In the following, we will describe the notation kits
that were devised for this study in detail.
The rule-based notation kit consisted of a 1-page manual detailing the notation as
laid out above, a pen, large sheets of paper, yellow sticky notes, and an adhesive tape.
Participants were instructed to use one sheet of paper per rule and think of meaning-
ful names same as those that would have to be used in an actual home automation
interface to identify the rule. Devices with triggers and actions were to be noted on the
sticky notes and placed on the large sheets of paper. The manual illustrated the rule
notation with an exemplary picture but also stated that the layout shown was merely
a suggestion. A translation of the manual can be found in Appendix A.1.
The process-oriented notation kit consisted of a 2-page manual detailing the notation
as laid out above, a pen, large sheets of paper, three colors of sticky notes (green, pink,
and orange), and an adhesive tape. Participants were instructed to use one sheet of
paper per process and think of meaningful names the same as those that would have
to be used in an actual home automation interface to identify the process. Devices with
triggers and actions were to be noted on green sticky notes, logic operators should be
noted on pink sticky notes, and for subprocesses, the re-used process’s name should be
written on an orange sticky note. All notes should be placed on the large sheets of paper
and connected with arrows to form an automation process. The manual illustrated the
notation with an exemplary picture but also stated that the layout shown was merely
a suggestion. A translation of the manual can be found in Appendix A.2. Instructions
were designed so that participants understood the presented notations but were neither
expressly instructed to follow them strictly nor forbidden to deviate from them.
Participants then—each, in case of a household with two participants—had to com-
plete one predetermined use case with the notation plus one to three of the ideas they
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:15
expressed during the home tour. Experimenters deliberately did not try to “correct”
the notation to match the instructive materials as it was also of interest to see how
participants would change their respective notation to make it their own. Deviations
were noted and discussed during appointment 2 in the semi-structured interview. To
conclude the appointment, experimenters would take pictures of the participants’ use
cases but leave the originals for the time being. Before leaving, they would encourage
participants to record more use cases until appointment 2 one week later.
4.2.3. Think Aloud and Interviews.
The beginning of appointment 2 then depended on
whether participants had recorded more use cases on their own. If so, they were asked
to explain their mental processes retrospectively. If not, they were asked to record
use cases using the think aloud approach. Participants could either decline or accept
this request. If they accepted, they had to come up with their own ideas and work
through the notation on their own. The goal of this section was to explore participants’
mental model while creating processes and rules. Two things were of interest there:
Is there a potential dissonance between the instructive material and the participants’
understanding of it and is the noted down use case the same as the one participants
originally envisioned? This phase was used to gather information for the subsequent
interview to unearth which observed effects were due to the differing notations and
which were due to misgivings in the mental model they constructed of the notation.
Following the think-aloud task, the semi-structured interview was conducted. The
interview dealt with the efficiency of the given notation for participants’ use cases first:
How fast did they feel they were in noting down single or complex use cases with their
notation? Could they come up with meaningful names? In what ways did the notation
support or not support clarity of overview for complex scenarios? Did participants feel
they were given enough information to work with the notation? Afterwards, condition-
specific questions were asked to elicit participants’ opinion on the elements specific to
their given notation and their opinion on the notation as a whole. If they had created
potential conflicts in their use cases, e.g., one rule that turns the lights on after 8 pm
and one that turns them off when no-one is home, they were asked if they were aware
of this and if so, what stopped them from solving the problem. The same was done if
they modeled potentially unintended side effects like one behavior closing the blinds
when the TV is on and one behavior to turn on the lights when the blinds are closed.
Subsequently, participants should evaluate whether they felt that the given notation
supported all their envisioned use cases. We were also interested to see whether par-
ticipants would use in-between steps, i.e., more than one order of trigger and action.
In the rule condition, this would constitute a violation of the notation that could lead
us to potential deficiencies of rule-based notations for home automation. Vice versa, if
participants in the process group saw no need for in-between steps, process notations
might be unnecessarily complex for automation scenarios. To conclude the interview,
general questions about home automation were posed: Do participants feel as if they
would use home automation in their everyday lives and what would their requirements
for home automation software be like? The complete interview script can be found in
Appendix B.
Afterwards, the recognition task was conducted. All participants were shown a use
case modeled by a different participant of the same condition, as well as one use case
from the other condition. In addition, an alternative of the process notation was shown
that lacked a starting point and placed the event information on the edges instead of
the device nodes. This notation came up in previous user testing with the homeBLOX
user interface [Rietzler et al. 2013; Walch et al. 2013], and we were interested in
how it would compare with the standard notations. An illustration of the adapted
process notation can be seen in Figure 4. With this task, we wanted to find out how
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:16 J.Brichetal.
Fig. 4. Example of adapted process notation.
well participants could reconstruct unknown use cases, both from the notation they
were familiar with as well as the respective other paradigm. The recognition task was
designed to gain insight into the understandability of each notation but also served
as a point of comparison where participants could discuss the benefits and drawbacks
they perceived in each notation. Use cases for process and rule notation were chosen
by the experimenters in each case from the available pool of use cases created by the
participants and transferred into a digital representation. Important in this case were
similar complexity and also, that use cases were chosen that did not stray too far from
the initially devised notations. The use case for the adapted process notation was “If
the TV is turned on, close the blinds. If then a movie is being played and someone leaves
the room, pause the movie until the person returns.”
Audio data was recorded all through appointment 2 and upon completion, experi-
menters took all recorded use cases and leftover material. Participants were thanked
for their participation and received the compensation. If the household consisted of two
participants, then appointment 2 was conducted with each in private.
4.3. Recruitment
We recruited potential participants through a number of channels, i.e., Twitter, Face-
book, and various university channels. Due to the potentially sensitive nature of the
study that required participants to invite the experimenters into their own home and
show them around, we also chose to recruit potential participants from our own social
circles to reduce that barrier. However, we ensured that, whenever logistically possible,
participants later would not be interviewed by the team member they knew personally.
We aimed for a diverse group of participants in our study showing different levels
of interest in home automation and technological affinity to be able to deduce what
characteristics home automation interfaces need to have to be appealing to the general
populace. To be able to select a balanced pool of participants for our study, we used a
demographic questionnaire on our pool of potential participants first. The characteris-
tics that were of interest to us were: Computer Science (CS) background (yes/no), age,
living situation (alone, shared apartment, or living as a couple), self-proclaimed level
of tech-savviness on a 5-point Likert scale, and self-proclaimed level of technophilia on
a 5-point Likert scale. Based on this, we chose pairs with similar characteristics from
the pool and randomly assigned them to the two conditions. Family households could
not be included in this sample as none volunteered. The specifics are detailed in the
following section.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:17
Table I. Overview of the 18 Participants and Their Respective Characteristics
ID Condition Background 乆/么Age Living situation Tech-savviness Technophilia
R24b Rule CS 么25 Shared 4 5
R24a Rule Not CS 么25 Shared 4 4
R31 Rule Not CS 乆26 Shared 1 4
R23 Rule Not CS 乆21 Shared 4 4
R29a* Rule Not CS 乆66 Couple 2 2
R29b* Rule Not CS 么65 Couple 4 5
R26a* Rule Not CS 乆23 Couple 3 3
R26b* Rule Not CS 么24 Couple 5 5
R27* Rule Not CS 么24 Alone 5 5
S19a Process CS 么25 Shared 5 5
S19b Process Not CS 乆22 Shared 3 3
S22 Process Not CS 乆23 Shared 2 2
S11* Process Not CS 乆22 Shared 4 4
S28a* Process Not CS 乆54 Couple 1 2
S28b* Process Not CS 么57 Couple 3 4
S30a* Process Not CS 么25 Couple 3 5
S30b* Process Not CS 乆28 Couple 1 5
S8* Process Not CS 么21 Alone 3 3
Starred entries denote participants that were interviewed by someone they knew personally.
4.4. Demographics
We chose 12 households with one or two members each to participate in our study. The
demographics of our 18 participants total are detailed according to the characteristics
mentioned before and can be seen summarized in Table I. The irregular participant
IDs are remnants from the recruitment pool.
Of the 18 participants, 9 were female and 9 male. Their age ranged from 21 to 67
years. All participants were German. Each group contained 1 one-person household,
4 participants living in a shared apartment, and 2 couples living together. The rules
condition comprised 4 women and 5 men with an average age of 33 years (Mdn =25,
SD =17.3). The process condition comprised 5 women and 4 men with an average
age of 31 years (Mdn =25, SD =13.4). Only one person in each condition had a CS
background, the others had academic as well as non-academic backgrounds. Regard-
ing tech-savviness and technophilia, participants in the rule condition had an average
savviness rating of 3.8 and technophilia rating of 3.8; participants in the process con-
dition had a savviness rating of 3.1 and technophilia rating of 3.0. Post-study analysis
showed that low technophilia or little interest in home automation did not have a sig-
nificant impact on participants’ engagement with the study in terms of the number of
modeled use cases.
Post-study analysis further showed that having a familiar interviewer did not signif-
icantly impact the number of use cases modeled in the study nor skewed participants
towards a certain opinion on home automation. Both groups gave equal praise of their
respective notation; however, participants that did not know their interviewer person-
ally were more reluctant to criticize the notations.
5. STUDY RESULTS AND DISCUSSION
In the following, we present our method of analysis and the results of the study clus-
tered by the respective research questions. Results are shortly discussed where appro-
priate, while an overarching summary of the resulting insights for home automation
interfaces is given in the next section.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:18 J.Brichetal.
Table II. Interview Categories Resulting From Coding
Use cases Home automation Notation
Comfort/home control (18) General interest, yes (18) / no (8) Criticism (18)
Security (7) Desire for automation (12) Praise (18)
Energy saving (6) Desire for remote control (8) Recognition task (18)
Pets (2) Desire for remote access (3) Deviations (14)
Plants (2) Desire for notifications (3) Problems with own use case (1)
Remote access (1) Preferred interface device smartphone (10)
Health (1) Preferred interface device tablet (9)
Other (3) Preferred interface device laptop (9)
Preferred interface device desktop (4)
Preferred interface device other (7)
Presumed software requirements (17)
Aspect concerning deployment (10)
Numbers indicate how many of the 18 participants made a mention in this category.
Table III. Device Types Participants Would Like to Automate
Lights Audio playback Central heating Stove/oven control Refrigerator
12 12 12 11 11
Window automation Coffee maker TV Smartphone
11 9 9 8
Numbers indicate how many of the 12 households mentioned this type.
5.1. Method of Analysis
From the home tours, we analyzed the device inventory sheets compiled for each house-
hold, as well as the collection of 249 freely formulated use cases to determine clusters.
With regard to the notation modeling, we analyzed 65 rule and 64 process recordings,
including use cases modeled at the end of the first session. We recorded 13 hours of
interview audio from the second sessions.
Qualitative analysis of the interview recordings was conducted by three coders in
an iterative coding process. Inter-rater reliability was determined using Fleiss’ Kappa
coefficient. The coding taxonomy was iteratively refined to obtain a taxonomy consistent
across groups. We iterated over one randomly selected rule and process participant each
to make sure categories were applicable for both. After each coding, the annotators
discussed and reconciled annotation discrepancies. Eventually, we settled on three
major categories: Use Cases (κ=0.70), Home Automation (κ=0.74), and Notation (κ
=0.60).
Corresponding subcategories and number of mentions can be seen in Table II.
In the following, we detail the results of our study with regard to potential and
acceptance of home automation as well as the comparison of notations.
5.2. Current Potential of Home Automation
Overall, households named on average 21 (SD =5.9) device types, e.g., lighting, heating,
media devices, and so on. During the hometour, households came up with 6–43 informal
use cases (avg. 20.8, SD =9.9) and 249 in total. As for the notation modeling task,
participants recorded 3–12 use cases on their own (rules avg. 7.2, SD =2.9; processes
avg. 7.1, SD =2.6) with 65 rule and 64 process recordings in total.
The device inventories revealed what device types participants would like to auto-
mate. The results can be seen in Table III. All households were interested in automating
lights, to either illuminate the house in the evening or when needed upon entering a
specific room or location within a room. Household 19 explained that their combined
home office/bedroom would benefit from selective lighting where one inhabitant could
go to bed and have this area darkened, while the other could continue working at the
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:19
Fig. 5. Distribution of devices that interviewers prompted in more than two households during home tours.
desk. Audio playback also played a dominant role with use cases focusing on media
streams following the user around the home or accompanying activities like relaxation
or cooking with characteristic music. S30b suggested the idea of creating a shower
setting where a relaxing light setting was complemented with the sounds of the rain
forest. Automated central heating was also mentioned by all households, either to sim-
plify thermostat routines like heating the bathroom before taking a bath or to save
energy more efficiently with heating schedules based on user presence. Eleven house-
holds wanted to automate windows, either to control the quality of air in the house with
regard to staleness and humidity or to make sure that all windows were closed when
no one was home. Unfavorable weather conditions like heat, rain, or strong winds were
also a concern. With regard to the kitchen, households mostly favored stove and oven
control, which would allow for automated cooking, and smart fridges able to manage
their contents and compile grocery lists. Interesting ideas were fridges capable of de-
tecting spoiled goods and a calories management system for the household. Automated
coffee makers were also high on the list, usually in combination with breakfast-making
routines triggered by inhabitants getting out of bed or finishing their bathroom routine.
Automating TVs received moderate attention with a variety of use cases; media stream-
ing was a topic, but also the use of the TV as part of a surveillance/door opening system
and also as an additional display for content otherwise displayed on smaller laptop
screens. Smartphones mostly played a role with regard to functioning as a universal
remote for the home or as a source of sensor and context information. As participants
mentioned that they would prefer notifications over full automation in some use cases,
the smartphone was deemed the best fit for receiving and displaying these in a timely
manner.
In addition to letting participants name devices on their own during the home tours,
we sometimes prompted them about whether common devices were to be found in
their home as well or when we could see devices that they had neglected to mention.
A distribution of the devices most commonly prompted can be seen in Figure 5. It
has to be mentioned though that from their reaction alone we cannot tell for sure
whether participants simply forgot mentioning a device or whether they neglected it
because they thought it was not suitable for home automation. However, participants
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:20 J.Brichetal.
were often very outspoken in telling us when they did not want a specific device to be
automated or could see no use in it. All in all–while not being conclusive–these results
can help indicate which devices participants currently associate intuitively with home
automation and which would need to be promoted more.
In general, participants showed a large interest in automating kitchen, bathroom
and cleaning equipment (96 of 249 mentions). We identified kitchen, bathroom, front
door, the living room couch, and pets and plants as respective activity centers of the
home using the approach devised by Crabtree et al. [2002]. This classification can help
designers find reusable automation solutions for common use cases aggregating around
these activity centers.
Participants especially liked the idea of automating devices for morning and coming
home routines. They showed comparatively little interest in automating entertainment
and access control (56). This contrasts starkly with the home industry’s current focus
on marketing automatable devices in this area like the seamless music solution Sonos1
or Link Interactive’s2door lock and glass-break monitoring.
In accordance with related work, participants did not aim for full-fledged building
automation but rather placed emphasis on everyday appliances like water heaters,
alarm clocks and coffee makers, which have largely been neglected by manufacturers
both with regard to automation and integration with the home. Since we did not re-
strict participants to use cases that were feasible with current technology, we expected
to encounter a number of futuristic ideas as well—especially from participants that
had little technical knowledge; however, we only observed few futuristic ideas, namely
“backpack packer,” “cereal preparer,” “mind remote control for TV and music,” “com-
pletely automatic shaving where I don’t have to do anything anymore,” and a fridge
that can move from one floor to the other to save inhabitants a trip down the stairs.
Overall, participants strongly favored comfort/home control use cases, e.g., automa-
tion that would spare them tedious, everyday tasks like turning things on or off, setting
alarms or pushing device buttons. For example, participants would like to automate
behavior such as “When I go to bed, remind me to charge my phone and set my alarm
according to my next-day appointments. As soon as I’m asleep, make sure all lights are
turned off and windows and doors are locked. When the alarm rings, open the blinds,
turn on lights in the bathroom and brew me a cup of coffee.” Participants were also of
the opinion that automation should help manage personal preferences in multi-person
households by tailoring for example lighting atmosphere, music taste, and room or
bath water temperature. Household S30 expected the smart home to be aware of their
preferences with regard to news interest, TV shows and music in order to present them
with a personalized media mix that should also take their current mood into account.
They also brought up the idea of interleaving their content streams when they were
both present. The two participants from shared household R24 also added that they
would like a smart home to help them schedule laundry and bathroom routines, espe-
cially when not only all inhabitants are present but also house guests. They imagined
a solution where the smart home would coordinate based on everyone’s waking time
and morning appointments. S19b would like the smart home to manage her extensive
library in a way that she could locate a book by searching for its title and having
the shelf indicate its position by a blinking light. Pet owners S19 and R27 would like
the smart home to take over the duty of feeding the animals. Plant owners would
generally want their plants to be watered automatically. Two participants also showed
concern for software updating needs. R29b wants device updates in the home to be
installed seamlessly, in contrast to, for example, his current navigation system that
1http://www.sonos.com/.
2https://www.linkinteractive.com/.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:21
needs to be taken out of the car, connected to the laptop to update, and then be brought
back. S19b remarked that she would need an exception for energy saving routines to
make sure that updating devices were not suddenly turned off when people left the
home. Participants also showed concern for others; R29a and S28a wanted the home
to automatically limit music or TV volume so as not to bother their partners, while
households S19 and R24 were concerned about bothering the neighbors. S30b in turn
wanted the home to help her take care of herself with regard to separating work from
free time by managing her calls. Private calls during work hours or work calls during
off time should be hidden, but notifications should be available upon explicit request.
Exceptions should be made for private emergency calls, which should always be patched
through until the user picks up the call.
Regarding energy use cases, the idea of “turn off appliances when we are away or
sleep” was paramount. Participants were however also concerned about charging their
devices. With regard to smartphones, many were worried about forgetting to do so,
while with electric toothbrushes, participants were aware that always putting them
back in the charger would damage the battery. Several participants would prefer to
have the smart home take care of these issues. 19a wanted the smart home to provide
him with detailed resource monitoring so that he could optimize his routines.
Participants had a general awareness of security issues; the prominent use case here
focused on supporting access control via identifying sensors or by employing a combina-
tion of cameras and smartphones to implement a remote door opener. Participants also
frequently brought up the need to be sure that automatic windows would reliably close
again to prevent break ins. Both R29 and S30 additionally wanted “vacation settings”
for blinds and lights that would make the home look inhabited even if the owners
were away on a trip. Participants also thought of appliance surveillance; household
S19 imagined an emergency stop and notification system for their washer in case of
flooding, while S22 wanted to keep tabs on her gas-powered water heater to prevent
dangerous carbon monoxide build-up. S30b also wants the smart home to take on an
educational role: In general, she wants to remember to turn off devices like stove or
lights by herself before she leaves the home; if she forgets, she wants the home to tell
her that she did, even if no emergency was detected.
Overall, 70.5% of the modeled use cases fell into the comfort/home control category,
followed by energy saving at 7.0% and security at 6.2%. These results should be mirrored
by both research and industry which have both hitherto put their focus more on energy
saving and security use cases and less on the convenience aspect of automation desired
by current home owners.
5.3. Current Acceptance of Home Automation
With regard to the current acceptance of home automation, we present our partici-
pants’ opinion on the perceived utility of home automation in general as well as their
requirements for potential home automation software.
5.3.1. Perceived Utility of Home Automation.
The majority of participants stated that they
would use home automation in their own home (15). Prominent reasons were “saves
time and frees me from tedious work” (9) and “is fun & entertaining” (3). R19a felt that
“it would be very comfortable and I would get to be lazy.” Also, S11 was enthusiastic:
“I have a ton of use cases.” Both S30b and 29a stated “It would make my life easier
and I would feel safer.” R27 liked the idea of both saving time and trips throughout
the house. However, participants also voiced reservations, e.g., due to associated costs.
Participant R29b noted that switching to home automation should not incur huge
financial or structural imposition. Four participants saw home automation at best as a
“nice to have.” Most participants stated at some point—and regardless of their general
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:22 J.Brichetal.
opinion of home automation—that they felt they did not really need it in their lives. S8
stated that his household did not warrant automation due to its small size; he could
however imagine things to be different with an increasing number of devices.
Reasons for wanting to use smart home technology were diverse in the areas Au-
tomation,Remote Control,Remote Access, and Notification. R29b summarized the ad-
vantages of home automation as (1) prevention of forgetfulness, (2) sparing people
unnecessary trips just to start or stop something, and (3) enabling things to happen in
the user’s absence. S30b noted that she had previously had no desire for home automa-
tion but after contemplating it for a week, saw large potential for it in her personal life.
Sixteen participants voiced specific automation needs, focused on comfort/home con-
trol, energy saving, and security in a way that matched results from the home tours.
Participants would also like systems to be able to recognize different users and execute
the corresponding set of preferences (music, lighting, temperature, etc.) when they are
on their own, e.g., relaxing in the living room. Several would like a generic remote con-
trol to automate devices in the home, while R31 emphasized the use of remote control
for entertainment purposes. S28a stated that a remote control would be her favorite
for interacting with electronic devices in the home. S28b saw the benefit in “being able
to control the home comfortably from the couch.” Four participants liked the idea of re-
mote access to the home’s digital infrastructure, for example, to remotely monitor and
control potential dangers like open windows or unattended stoves, or to request status
messages from devices like the fridge or the answering machine. R26b found the idea
of combining remote control capabilities with remote access to the house’s functionality
“amazing.” Eight participants preferred notifications over automation, while three em-
phasized the importance of memory assistance, e.g., for reminding people about wallets
or keys when leaving the home. Another three participants stressed that they wanted
to be informed in case of potential security problems at home.
Statements against home automation (12) focused primarily on the notion that au-
tomation was unnecessary because tasks could be completed without. S11 for instance
stated quite energetically “Some things [...] would just be silly. I can really do that on
my own!” Participants often felt that things like flicking a switch just did not warrant
the necessary overhead to create a smart home. Two participants also observed that
their lives were more exception than regularity and could not imagine them expressed
as automation behavior. S22 worried about forgetting “the non-automated bits” like
refilling the water tank of the plant watering system. She felt that notifications were
a better fit for her. Other rejections were grounded in the fear of becoming lazy, losing
control, or a general dislike of technologized environments. S28a felt that an auto-
mated home would be “too sterile, too mechanical” for her. S30b disliked automatic
decisions and would prefer a system of notifications and explicit user interaction. The
participant described a scenario as critical where automation was set up for guests; in
case of a break-in the automatic system might treat the intruder as a guest whereas
the notification system would attract the inhabitant’s attention to the intruder. R24a
however conceded that contrary to his general dislike of home automation, he would
“probably use it if it was installed anyway.”
Participants also commented on perceived real-world deployment issues. S19b was
worried that automation behavior intended for the inhabitants might accidentally be
triggered by their dog. Conflicts and side effects, especially in scenarios with several in-
habitants, would cause significant impairment and deduct from the overall usefulness
of home automation. The preferred system behavior would be automatic correction or
compromise, however participants conceded that a purely technical solution might not
be feasible. S30b would like the automation system to be able to detect potential con-
flicts on its own and get back to the user to ask for help if that was the case. S19b and
R31 would choose a pre-deployment collaborative approach with their cohabitants to
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:23
manually balance automation desires of inhabitants and avoid conflicts later on. Five
participants stated that they would be willing to correct or adapt automation specifi-
cations manually when noticing unwanted automation effects in the home’s behavior.
S30a felt that a thorough “debugging” phase of automation behavior might lead to ad-
ditional work in the beginning but also improve the system’s performance in the long
run. R29a stated that depending on the severity of conflicts and side effects, she “might
not make the effort” to correct the behavior and instead learn to live with it. The idea
of technology not working as expected however frustrated her immensely.
All in all, participants supported the idea of home automation but were not overly
convinced that home automation systems would work in a stable and predictable man-
ner in real-world scenarios. This is an image issue both deeply rooted in end users’
previous experiences with home electronics as well as their lack of knowledge of the
inner workings of smart technology. For home automation to succeed in the general
market, end users will have to be convinced that allowing additional technology into
their homes will be more than an integration and device management nuisance.
5.3.2. Home Automation Interfaces.
With regard to home automation interfaces, 14 par-
ticipants would like to configure their homes manually. Two would only want to use
configuration software “if it was really easy,” and R29b preferred to have a remote
control with programmable codes. R29a conceded that she would probably not use the
software on her own, but rather ask her partner to do it for her.
When asked about requirements for a proposed user interface, participants came up
with an abundance of ideas. With regard to first steps, six participants said that the
picture of the notation example in the introductory material of the notation kit had
been the biggest help. Four liked being tutored by the experimenter, three found the
introductory text in the instruction sheet helpful. R23 however stated that the text was
too long and too complicated. S8 expressed the need for a more detailed explanation of
context events and special functions. This suggests the need for a variety of support
levels and modalities in home automation interfaces.
With regard to clarity, participants remarked that colors helped to discern various
elements (5); that a configuration interface has to be easy to use without programming
experience (4); that devices, actions, events, or rooms need to have expressive names
or should be nameable by the user (4); that icons instead of text would be helpful (four,
in contrast to two participants preferring text), while two participants each favored
easy (de-)activation of stored rules or processes, high clarity, and the abstraction from
technical details. In addition, participants from the rule group stated that they did not
expect large rule sets to get confusing after creating use cases. One however was of
the opinion that he would only define one rule per device while another stressed the
necessity to group related rules, e.g., for lighting or heating controls. S8, R24b, and
S22 declared that a rule-based interface would not be sufficient for their automation
needs: “If you’re getting home automation, you want to do something complex. This
notation [rules] would upset me.” (S8) and “complex configurations should definitely be
possible.” (R24b and S22). Further desired characteristics included editing and updat-
ing previously defined rules or processes (3), freedom with regard to modeling direction
on the canvas and the order of configuration steps (2), sufficient security (2) and pri-
vacy (1), and consistency across several devices (1). Two participants emphasized the
importance of a visually appealing look. Interesting suggestions further included auto-
matic detection of modeling mistakes, a simulation mode to test automation behavior
before actual deployment, the ability to use standard functions like “turn device x
on/off” in the form of templates that users could then extend, and intervention options
to reset automation behavior at runtime.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:24 J.Brichetal.
When asked about the preferred device for smart home configuration, some partic-
ipants explicitly preferred a laptop (4) or general computer (5) based software, since
they already owned one or because it provides more structure and clarity compared
to a mobile device (five in favor of laptop and three in favor of a PC). For R24a and
R26b, this was the only conceivable option, under the premise that they would use a
home automation interface at all. While a tablet was considered optimal by a larger
group (8), participants would not want to buy one just for home automation (5). Three
participants would like to have a dedicated tablet for home automation so that all
inhabitants could share it without having to share someone’s personal device. Nine
participants could see advantages in utilizing a smartphone. However, six stated that
the display size was too small. They advised to use the smartphone for special tasks
(mostly remote access) in conjunction with one of the other options. S22 saw a benefit
in using the smartphone for quick check-ups that do not merit the boot time of other de-
vices. S30b expressed concern that a smartphone could be seized too easily by strangers
and compromise home security. Notions beyond well-established devices included mind
control (S11), a remote control with display (S11), touch-based wall displays (S28a and
R29b) and hard-wired control boxes (S28b and R26a). Overall, participants stressed
form factors that provided clarity and adequate display size; also, they preferred touch
interaction.
5.4. Rule-Based vs. Process-Oriented Notation
This section details the results from the analysis of participants’ use of the two notation
variants.
5.4.1. Complexity of Modeled Automation Tasks.
On average, the rule group used 3.3 de-
vices per use case, while the process group used 4.5. A Mann–Whitney Utest (U=14,
z=−2.3, p<0.05, r=−.55) shows that the device count per use case was significantly
lower in the rule group (M=2.8) than in the process group (M=4.1). Device count
can be seen as an indicator of a use case’s complexity. In this case and based on our
results, it can be said that participants from the process group tended to model more
complex behaviors than participants from the rule group.
Participants however also exhibited diverse modeling behaviors. The process group
modeled automation behaviors with dedicated devices as instructed, while the rule
group tended to describe vague contexts on a single sticky note instead of following the
device (event) pattern for the trigger column. Therefore, we further evaluated the simple
behavior count, which we defined as the number of behaviors with only one trigger and
exactly one action. The rule group overall modeled 17 simple behaviors (26.2%, M=
1.0). In contrast, the process group only modeled 4 simple behaviors (6.3%, M=0).
Using a Mann–Whitney Utest, this constitutes a significant difference (U=18.5,
z=−2.1, p<0.05, r=−.49).
Both these metrics show that participants in the process group modeled more com-
plex behaviors than the rule group, which supports the hypothesis that processes are
more expressive.
Additionally, the process group had the opportunity to reuse already recorded pro-
cesses as subprocesses. Yet, only two participants made use of this concept and one of
them used subprocesses simply as triggers for other processes. The rule group was not
primed to reuse already modeled behaviors; nonetheless, R29b modeled similar behav-
ior for a vacation use case: “If on vacation, a ‘show signs of habitation’ (e.g., shutter-
and light-control) behavior should be activated.”
5.4.2. Participants’ Perceptions of the Notations.
During the semi-structured interviews,
participants could comment on their respective notations. It has however to be noted
that these perceptions relate to the notation as used by individual participants, which
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:25
partially deviated from the given notation as detailed in the following section. Eight
participants in the process group and four in the rule group indicated that their no-
tation allowed for neatly arranging simple use cases; six of them confirmed this for
complex processes, and three for complex rules. Rules were described as simple and
straightforward by eight participants, processes by four. Both notations were perceived
as efficient (seven from the process group, six from the rule group). Six participants
described processes as intuitive and easy to understand, and two expressed the same
for rules. The rules were considered as structured by four participants, while one par-
ticipant voiced the same for processes. Five participants of the process group and three
of the rule group stated they saw no restrictions in their notation. The statements of
the rule group were weakened in two cases by adding the comment that this would
involve “tremendous effort.” These perceptions indicate that rules are sufficient for
simple tasks but become confusing for more complex use cases, while processes remain
comprehensible.
When asked to criticize their notations, half the rule group (4) stated they felt re-
stricted by their notation. Two participants further described the rule notation as in-
flexible, confusing and uncomfortable for more complex use cases. Two participants
described processes as confusing when lines crossed, difficult, or unintelligible;2noted
that they lacked understanding concerning the logical joins. This indicates that both
notations have potential drawbacks that need to be addressed in interfaces planning
on employing these paradigms for home configuration.
We further asked participants to suggest improvements for their respective notation.
In the process group, three participants felt that a defined start point is unnecessary. It
was also mentioned twice that no arrows would be needed and lines would be sufficient,
as directionality (derived from the western reading system from left to right) was
intuitively clear. Individual suggestions were given about the design: conditions could
be annotated on the lines, and the use of color or different types of strokes could help
to distinguish one path from another.
In the rule group, four participants stated they would like to be able to group rules
in different ways, e.g., by device, function, or room; three also reported that they would
like to model temporal dependencies. They suggested to introduce an “after” column,
to define order based on the vertical position, or to offer the possibility to chain rules.
5.4.3. Errors and Deviations From Notation.
To assess how well participants could apply
the given notations, we analyzed their created use cases for errors, inconsistencies,
and deviations from the notation. Since participants were not restricted in their use of
the given notation, errors can be interpreted as indicators for deviations between the
recorded use case and what the participant had in mind. Deviations that clearly stray
from general semantics are also seen as errors. In the process group, two participants
had issues with the AND and OR semantics. This was to be expected as logic operators
are an abstract construct whose application needs to be explicitly learned. In one case,
joins were drawn with only one inbound arrow modeled, understood as sequential
execution. Another participant misunderstood the semantics of arrows, and modeled
start events that were independent from one another as a process. In the rule group,
two participants did not grasp the separation of triggers and actions. For example,
actions were entered into the list of triggers and vice versa.
We did not identify any unintentional side effects between the created use cases in
either group. This was probably due to the relatively closed-off character of the use
cases provided by participants with few overlaps and the relatively low number of
modeled use cases per participant.
Distinct patterns however emerged in the participants’ deviations from the given
notation. In the rule group, one deviation could be observed for all participants:
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:26 J.Brichetal.
Fig. 6. An example how lines were added to the rule notation (participant R26b).
Fig. 7. A created rule as an example of how participants changed the notation (participant R26a).
Despite being instructed to use one sheet of paper per rule, rules that were considered
to belong together were always modeled together, i.e., one below the other. A related
variation was the use of arrows or lines to connect triggers and actions (which was
not introduced as part of the notation), which were used to further distinguish
between individual rules on a sheet with more than one rule (five participants).
One example of this is shown in Figure 6. Observed difficulties related to the use of
AND and OR operators (four participants), which were modeled as single elements
or stated as a condition next to an arrow. In three cases, temporal and hierarchical
dependencies were modeled which were represented by arrows or lines. See Figure
7 for an example of a more complex rule. These deviations show that participants
of the rule group adapted their notation to express more complex use cases leaning
toward the process-oriented notation—without being formally introduced to that
notation paradigm. These results support the hypotheses that some real-world home
automation use cases require capabilities beyond those of a strict rule-based notation.
Four participants had difficulties in representing OR semantics. These participants
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:27
Fig. 8. Participant S30a modeled solid (1), dashed (2), and dotted (3) stroke types to distinguish the paths
of the process; strokes were digitally redrawn for clarity.
were mostly unwilling to add additional rules (3) stating “it HAS to be possible to do
it like this,” while one did not think of modeling an OR relationship as multiple rules.
For the process-oriented notation, the start and end points of processes were repeat-
edly omitted by two participants; instead, the beginning of the process was perceived as
its start, and the respective last actions as well-defined ends. In one case, arrows were
entirely omitted, as it was assumed that the customary reading direction would suffice.
Two participants tried to distinguish different paths of the process by using distinct
types of strokes, an example of which can be seen in Figure 8. The implicit AND and
OR-split, which was realized by several outgoing edges of a node, was explicitly used by
one participant as a separate element. Elements without preconditions were modeled
without an incoming edge by two participants. Two participants in this group were
also unsure when to use the AND or the OR join during process creation.
5.4.4. Interpretation of Others’ Use Cases.
In the final recognition task, all 18 participants
were shown a use case modeled by a different participant of the same condition, as well
as one use case from the other condition. In addition, an alternative of the process
notation was shown. The respective use cases were generally well interpreted. There
were no differences between the respective notations, indicating that it is possible to
interpret these notations even without prior knowledge.
These results allowed us to draw a number of insights that can be useful when
designing an end user programming interface for home automation. The insights are
presented in the following section.
6. INSIGHTS REGARDING END USER PROGRAMMING INTERFACES IN HOME AUTOMATION
Since smart home technology aims to be ubiquitous and unobtrusive, any kind of
interaction should be tailored especially to less tech-savvy users. This notion conflicts
with our participants’ desire to understand and control their immediate environment
to the fullest extent. Home automation interfaces will have to find a way to mediate
this conflict. Our findings show that process-oriented notations for configuration could
be a viable approach to achieve that. Davidoff et al. [2006] however state that strict
routine models are insufficient for end users as domestic routines are subject to change
over time; therefore, we propose to extend interface implementations in the fashion of
Koskela et al.’s “instant control” [Koskela and V¨
a¨
an¨
anen-Vainio-Mattila 2004], e.g., by
additionally incorporating remote control aspects.
Our results indicate that while strict rule-based notations are sufficient for sim-
ple automation tasks and are perceived as well structured, participants from the rule
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:28 J.Brichetal.
group felt restricted when considering more complex automation tasks. In relation to
that, Garc´
ıa-Herranz et al. [2010] point out that limited capabilities could stress users.
Participants provided us with real-life use cases that they could not model with the rule-
based notation without extending it with features reminiscent of the process-oriented
notation: grouping of rules, line connections, and temporal as well as hierarchical de-
pendencies. Therefore, we propose to provide more expressive process-oriented notation
for systems that are conducive of more complex automation tasks. Home automation
interfaces should offer both simple and complex modeling options in order to capitalize
on both clarity and expressiveness, as also suggested by Ur et al. [2014]. This could
be achieved by allowing for simple rules to configure simple automation tasks with
few triggers and actions, while a process-oriented approach could extend on this to
support complex tasks with temporal dependencies. Alternatively, interfaces could of-
fer tools and assistance for seamlessly expanding rules into processes. In our notation
study, participants from the rule condition intuitively combined the two approaches in
such a fashion by extending the rules notation with the additional features mentioned
above—depending on the requirements of specific use cases.
Concerning end user configuration in general, Newman et al. [2008] state that de-
signing for future states is easy for programmers, yet introduces a certain level of
insecurity in end users. Several approaches [Newman et al. 2008; Dahl and Svendsen
2011; Leitner et al. 2013] support the notion to provide a simulation mode for home
automation interfaces that was also voiced by our participants. Simulation modes pro-
vide a safe environment where users can get to know automation technology or try
out new ideas. The difficulty with this will be how to represent the potentially large
number of devices in a household in a comprehensive manner. To be meaningful, sim-
ulation modes would have to reflect the individual home situation to at least a certain
degree—by mirroring the floor plan or representing a throng of devices with different
capabilities. Light automation alone would require a number of devices positioned all
across a room, all of which can differ with regard to brightness, color, area of illumi-
nance, and dimming capabilities. The amount of data increases exponentially as soon
as several rooms or more device types should be considered for automation. It is yet un-
clear how home automation interfaces could acquire this information without requiring
the user to manually map their entire home. And even if users would be willing to do
this once, it has to be kept in mind that home arrangements change over time with
devices being replaced or rearranged. We envision that home interface designers will
have to draw upon visualization techniques that were developed for depicting large
datasets in other domains to make this amount of data manageable and intuitive to
understand for end users.
Furthermore, the strong desire for a veto right to automatic adaptations also confirms
prior work [Koskela et al. 2004; Davidoff et al. 2006; Mennicken and Huang 2012b]. In
a home automation interface, this could be realized in various ways. One of our par-
ticipants stated that he would be satisfied if the system allowed him to turn the lights
back off after the automation had determined to turn them on. On part of the software,
this does however require additional logic and some sort of memory. Otherwise, the
system would just revert back to the automated settings the next time conditions were
evaluated. Along a similar notion, haptic interaction could be conceived; for instance,
users could place a decorative element in a certain position to override automation for
a specific room. This could also be used to solve conflicts, e.g., one inhabitant could
declare priority over the others in terms of current media choices or room settings.
Another participant would like the system to ask questions if unsure how to proceed;
this can however raise the issue of having to find a satisfactory compromise between
smart ubiquity and continuously prompting the user.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:29
The number of deviations from the given notation also indicates that due to the
complex nature of some automation tasks, home automation interfaces need to provide
extensive assistance, especially for initial use. As expected, we observed a need for
support especially for more advanced features of the process-oriented notation, i.e.,
logic operators. However, our results also indicate that one-size-fits-all solutions would
likely not be effective due to our participants’ different ways of approaching configu-
ration tasks. Instead, diverse assistance should be provided, such as instructions at
different levels of granularity and software support for certain notation features, such
as logical joins. Here, ideas can be drawn from classical software engineering. Intro-
ductory information can be presented as text, video or step-by-step clickable tutorials.
Information shown in the beginning should however always be retrievable; either as
an integrated compendium or in the form of tool tips. Our usability testing in previ-
ous work also revealed that configuration interface users employ different interaction
strategies and appreciate the redundant implementations of features, for example,
that a device icon can be deleted via an explicit garbage can area or simply by being
dragged off the screen [Rietzler et al. 2013; Walch et al. 2013]. Senior users also voiced
additional requirements, such as adjustable font size and zoom capabilities.
With regard to the general acceptance of home automation, participants voiced con-
cerns over feasibility, cost, reliability, and security. Since home automation operates
in a user’s personal territory, concerns for security and privacy are understandable
and valid. Building user trust by informing inhabitants about the underlying rationale
of the home automation system, offering effective veto options and making sure that
private information is sufficiently protected will be paramount to the success of home
automation technology. In addition, feedback from participants living in multi-person
households suggests a need for moving away from the single-user paradigm which is
often assumed as a default setting in home automation technology. While easier to
deal with, it is not representative of real-world user requirements. Home automation
interfaces will need to find a way to deal with conflicts and offer strategies to arbitrate
between the preferences of multiple users.
7. STUDY LIMITATIONS
A potential limitation of our study is the sample size of 12 households and 18 partic-
ipants. However, it provides a sample of typical size for qualitative studies and our
results provide rich insights on home automation needs and the design of configura-
tion interfaces. We further aimed to balance our groups in terms of their demographics
and recruited participants with diverse living situations, i.e., single households, shared
flats, and couple households. Although the rule group showed slightly higher techni-
cal interests and skills, we could not find indications that it influenced our results.
While our participants were likely not fully representative of the general population,
especially since family households were missing from the sample, we believe that our
results carry valuable insights for the design of home automation interfaces that are
likely generalizable beyond our sample population and should be validated for different
demographics. However, we have to point out that all our participants came from the
same cultural background. Results may vary for different cultural settings.
Concerning the study design, participants provided use cases with a wide range
of complexity; therefore, we conclude that the instructive examples did not prime
participants toward a certain complexity.
A limiting factor is that participants merely thought about potential use cases rather
than using real-world home automation. While they did this in the context of their own
home, participants still could only have a limited idea of what living with home automa-
tion would truly look like. Coming into contact with the actual benefits and drawbacks
of the technology could change their perception of it. Furthermore, implementations
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:30 J.Brichetal.
of home automation configuration interfaces will presumably never be able to accom-
modate all mental models or configuration strategies of which our participants have
likely just shown a few; this will lead to usability issues that will have to be studied
for each solution. This holds especially true when concepts from the two paradigms
we researched separately are to be combined. Furthermore, some participants created
only a few use cases during the week between assignments; however, the majority pro-
vided multiple use cases, which suggests that they took our assignment seriously and
were engaged in the study. It has to be kept in mind though that this does not allow for
making any assumptions about how seriously they would engage with real-world home
automation which as-of-yet is a lot harder to set up and maintain than a piece of paper.
Overall however, our results suggest that not bringing them in contact with actual
home automation technology allowed them to be more creative in how they approached
home automation and potential use cases for it, which matched our intention of elicit-
ing their everyday requirements with no regard for current technological constraints.
The insights we gained in this study can thus be a first step in the design of home
automation interfaces to better support end users in the programming of their homes.
8. CONCLUSION AND FUTURE WORK
Home automation faces the challenge of providing ubiquitous, unobtrusive services
while also supporting end users with approachable programming interfaces. As these
interfaces further need to provide sufficient expressiveness to support complex automa-
tion behaviors, proper notation styles for less tech-savvy users need to be devised. Over
the last years, rule-based and process-oriented paradigms have emerged; however,
their underlying concepts had not been studied in a comparative manner. We reported
on a contextual inquiry study that collected qualitative data from 18 participants in
12 households in their own homes. We gathered information on the current potential
and acceptance of home automation in general, as well as explored the benefits and
drawbacks of the two programming notation paradigms for end users. Our results
show that rule-based notations are sufficient to express simple automation tasks but
can be limiting for more complex use cases. Participants that used the process-oriented
notation valued its expressiveness and presented us with more complex use cases in-
volving more devices in their homes. We identified benefits and drawbacks of the two
paradigms that suggest that providing end users with a mix of the two approaches
might be beneficial to address a broad variety of automation desires. We discussed how
our results can inform the design of end user configuration interfaces for smart home
technology in order to enable usable real-world home automation for end users.
As a next step, approaches that bridge the analog–digital divide could be used to
provide users with small-scale functional home automation that still retains the ben-
efits of intuitive, experimental pen and paper interaction. Hess et al. [2012] describe
an approach for business process modeling where informal pen input can be combined
with a rudimentary software, providing feedback similar to full-scale solutions. In the
home context, this could lead to prototypes that enable configuration input in the form
of sketched rules or processes that are then transferred into actual sensing and actua-
tion commands in the home. Compared to pen and paper approaches, this would allow
participants to get a better grasp on what living with home automation will actually
be like and improve their assessment. Researchers could leverage this to get new in-
sights regarding programming metaphors without the need to develop fully functional,
high-fidelity interfaces for the participants to interact with the home automation pro-
totypes. With regard to developing interfaces, once promising configuration strategies
have been identified, McCurdy et al.’s [2006] approach of mixed-fidelity prototypes can
be used to break down the task of developing complex automation software into man-
ageable units by iteratively focusing on various dimensions of interest. Transitioning
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:31
from the fully analog to the fully digital approach with these intermediary steps will
help to ensure that the resulting interfaces for the home automation market fulfill
end user requirements on a number of different levels. Traditional usability testing
can then be used to evaluate how successful each interface solution is in conveying its
underlying programming paradigm to the user.
APPENDIXES
A. NOTATION KIT MANUALS
The manuals given to participants of the study to detail the characteristics and in-
tended workflow of the respective notation are presented here. The materials were
originally devised in German and are presented in translation.
A.1. Instructions for Rule-Based Notation
—Use one sheet of paper per rule.
—Put a meaningful name for the rule in the upper left corner.
—Rules consist of triggers (stemming from events) and resulting actions. Both triggers
and actions are assigned to devices. Devices should be written down on the sticky
notes and put on the paper.
—A rule is executed when all triggering events happen. To use a device as trigger, you
need to specify its appropriate event, e.g., the alarm clock has the event “ringing.”
—To execute an action, you need to specify the desired action on the device sticky note,
e.g., coffee maker has the action “make one cup of coffee” or alarm clock has the
action “set alarm,”
—Additionally, you may use a timer as device to model temporal dependencies. To do
this, you may specify a rule A where the action “timer(set ×minutes)” is executed
after a given event and another rule B where “timer from rule A elapsed” is used as
triggering event.
—An example for a rule is “if the alarm clock rings and it is between 6 and 8 am, then
make one cup of coffee.” (This was illustrated with a picture similar to Figure 2.)
—The layout shown in the picture is presented merely as a suggestion.
A.2. Instructions for Process-Oriented Notation
—Use one sheet of paper per process.
—Put a meaningful name for the process in the upper left corner.
—Every process consists of interconnected devices; devices should be put on green
sticky notes. Each device has events and actions, e.g., alarm clock has the action “set
alarm” and the event “ringing.” Connect devices with arrows and note what events
trigger what actions.
—Every process has a designated starting point. Arrows start at that point and connect
to the first order of devices. More devices can be added by adding more arrows.
—To execute an action, you need to specify the desired action on the device sticky note,
e.g., coffee maker has the action “make one cup of coffee.”
—To combine events and actions, you can put “and” and “or” on pink sticky notes and
place them at the appropriate position in your process; “and” means that all events
need to happen to execute an action, “or” means that one of them suffices.
—Additionally, you may use a timer as device to model temporal dependencies. You can
incorporate this element in your processes the same way you would include a device.
For this, note the time that you want to postpone the following action execution for.
—You can reuse already recorded processes as subprocesses. For this, note the name
of the previously recorded process on an orange sticky note and incorporate this in
your new process where appropriate.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:32 J.Brichetal.
—An example for a process is “if the alarm clock rings and it is between 6 and 8 am,
then make one cup of coffee.” (This was illustratedwith a picture similar to Figure 3.)
—The layout shown in the picture is presented merely as a suggestion.
B. INTERVIEW SCRIPT
The script of the semi-structured interview is detailed in the following. The questions
were originally devised in German and are presented in translation.
—Efficiency of the notation for use cases: How fast could you create processes/rules for
single and complex use cases? Could you come up with meaningful names?
—Clarity of overview: If so, what did you like? If not, what was problematic? How was
clarity for simple processes/rules? How for complex ones?
—Were you provided enough information to work with the notation? What helped,
what did you miss?
—(Condition-specific) Did you encounter difficulties working with the notation? (ask
about all elements)
—What did you like about the system?
—What did you dislike about the system?
—Where there functionalities you missed?
—(if they created potential conflicts in their use cases) Were you aware that this might
cause problems? If yes: What were the reasons for not rectifying it (did not know
how, did not find it necessary, ...)?
—(if they modeled potentially unintended side effects) Were you aware that this would
trigger behavior other than intended? If yes: Would that bother you? If yes, what
were the reasons for not rectifying it? If no, why not?
What would you need from the system in terms of support to handle these situations?
—Did you feel that the notation supports all of your envisioned use cases? If yes: how
does it support? If not: how does it inhibit?
—In-between steps:
If used: Why and what for? (in case of rule condition: Did you explicitly ignore the
given notation?)
If not used: Why not? (in case of rule condition: Because it was not in the given
notation or because it did not feel necessary?)
—Do you feel as if you would use home automation in your daily life? If so: Why? If
not: Why not?
—Do you feel as if you could use this system in the form of an automation software?
Why/why not? What would you wish this software to be like? What device would you
like to use that software on?
ACKNOWLEDGMENTS
We would like to thank all our participants for inviting us into their homes.
REFERENCES
Frances Aldrich. 2003. Smart homes: Past, present and future. In Inside the Smart Home. Springer, 17–39.
Christopher Alexander, Sara Ishikawa, and Murray Silverstein. 1977. A Pattern Language: Towns, Buildings,
Construction. Oxford University Press.
Lyn Bartram, Johnny Rodgers, and Rob Woodbury. 2011. Smart homes or smart occupants? Supporting
aware living in the home. In Proceedings of Conference on Human-Computer Interaction (INTERACT’11).
Springer, 52–64.
A. J. Bernheim Brush, Bongshin Lee, Ratul Mahajan, Sharad Agarwal, Stefan Saroiu, and Colin Dixon.
2011. Home automation in the wild: Challenges and opportunities. In Proceedings of CHI’11. ACM,
2115–2124. DOI:http://dx.doi.org/10.1145/1978942.1979249
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:33
Marie Chan, Daniel Est`
eve, Christophe Escriba, and Eric Campo. 2008. A review of smart homes—Present
state and future challenges. Comput. MethodsProg. Biomed. 91, 1 (2008), 55–81.
Ming Ki Chong, Rene Mayrhofer, and Hans Gellersen. 2014. A survey of user interaction for spontaneous de-
vice association. ACM Comput. Surv. 47, 1 (May 2014), Article 8, 40 pages. DOI:http://dx.doi.org/10.1145/
2597768
Control4. 2016. Control4 App. Accessed April 5, 2016 from http://www.control4.com/solutions/products/
control4-app.
Enrico Costanza, Joel E. Fischer, James A. Colley, Tom Rodden, Sarvapali D. Ramchurn, and Nicholas R.
Jennings. 2014. Doing the laundry with agents: A field trial of a future smart energy system in the home.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 813–822.
Andy Crabtree, Terry Hemmings, and Tom Rodden. 2002. Pattern-based support for interactive design in
domestic settings. In Proceedings of the 4th Conference on Designing Interactive Systems: Processes, Prac-
tices, Methods, and Techniques (DIS’02). ACM, 265–276. DOI:http://dx.doi.org/10.1145/778712.778749
Yngve Dahl and Reidar-Martin Svendsen. 2011. End-user composition interfaces for smart environments:
A preliminary study of usability factors. In Proceedings of Conference on Design, User Experience, and
Usability (DUXU’11). Springer, 118–127. DOI:http://dx.doi.org/10.1007/978-3-642-21708-1_14
Scott Davidoff, MinKyung Lee, Charles Yiu, John Zimmerman, and Anind K. Dey. 2006. Principles of
smart home control. In Proceedings of International Conference on Ubiquitous Computing (UbiComp’06).
Springer, 19–34. DOI:http://dx.doi.org/10.1007/11853565_2
Luigi De Russis and Fulvio Corno. 2015. HomeRules: A tangible end-user programming interface for smart
homes. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in
Computing Systems. ACM, 2109–2114.
Anind K. Dey, Timothy Sohn, Sara Streng, and Justin Kodama. 2006. iCAP: Interactive prototyping of
context-aware applications. In Proceedings of International Conference on Pervasive Computing (Perva-
sive’06). Springer, 254–271. DOI:http://dx.doi.org/10.1007/11748625_16
Colin Dixon, Ratul Mahajan, Sharad Agarwal, A. J. Brush, Bongshin Lee, Stefan Saroiu, and Paramvir Bahl.
2012. An operating system for the home. In Proceedings of USENIX Symposium on Networked Systems
Design and Implementation (NSDI’12). USENIX Association, 25–25.
e-Domotica. 2016. Homepage. Accessed April 5, 2016 from http://www.e-domotica.com/en/.
Roland Eckl and Asa MacWilliams. 2009. Smart home challenges and approaches to solve them: A practical
industrial perspective. In Intelligent Interactive Assistance and Mobile Multimedia Computing.Springer,
119–130.
Manuel Garc´
ıa-Herranz, Pablo A. Haya, and Xavier Alam´
an. 2010. Towards a ubiquitous end-user program-
ming system for smart spaces. J. UC S 16, 12 (2010), 1633–1649.
Richard Harper. 2003. Inside the smart home: Ideas, possibilities and methods. In Inside the Smart Home,
Richard Harper (Ed.). Springer, 1–13. DOI:http://dx.doi.org/10.1007/1-85233-854-7_1
Jan Hess, Christian Reuter, Volkmar Pipek, and Volker Wulf. 2012. Supporting end-user articulations in
evolving business processes: A case study to explore intuitive notations and interaction designs. Int. J.
Coop. Inf. Syst. 21, 04 (2012), 263–296. DOI:http://dx.doi.org/10.1142/S0218843012500049
HomeSeer. 2016. HSTouch. Accessed April 5, 2016 from http://www.homeseer.com/hstouch-mobile-app.html.
Jan Humble, Andy Crabtree, Terry Hemmings, Karl-Petter Åkesson, Boriana Koleva, Tom Rodden, and P¨
ar
Hansson. 2003. “Playing with the bits” – User-configuration of ubiquitous domestic environments. In
Proceedings of International Conference on Ubiquitous Computing (UbiComp’03). Springer, 256–263.
DOI:http://dx.doi.org/10.1007/978-3-540-39653-6_20
Amy Hwang and Jesse Hoey. 2012. Smart home, the next generation: Closing the gap between users and
technology. In Proceedings of the Fall Symposium on Gerontechnology. AAAI.
IFTTT. 2016. Homepage. Accessed April 5, 2016 from https://ifttt.com/wtf.
Insteon. 2016. Homepage. Accessed April 5, 2016 from http://www.insteon.com/.
Michael O. Jewell, Enrico Costanza, and Jacob Kittley-Davies. 2015. Connecting the things to the internet: An
evaluation of four configuration strategies for wi-fi devices with minimal user interfaces. In Proceedings
of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’15).
ACM, New York, NY, 767–778. DOI:http://dx.doi.org/10.1145/2750858.2807535
Eija Kaasinen, Tiina Kym¨
al¨
ainen, Marketta Niemel¨
a, Thomas Olsson, Minni Kanerva, and Veikko Ikonen.
2012. A user-centric view of intelligent environments: User expectations, user experience and user
role in building intelligent environments. Computers 2, 1 (2012), 1–33. DOI:http://dx.doi.org/10.3390/
computers2010001
Thomas Kleinberger, Martin Becker, Eric Ras, Andreas Holzinger, and Paul M¨
uller. 2007. Ambient intel-
ligence in assisted living: Enable elderly people to handle future interfaces. In Universal Access in
Human-Computer Interaction. Ambient Interaction. Springer, 103–112.
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

11:34 J.Brichetal.
Tiiu Koskela and Kaisa V¨
a¨
an¨
anen-Vainio-Mattila. 2004. Evolution towards smart home environments: Em-
pirical evaluation of three user interfaces. Pers. Ubiq. Comput. 8, 3–4 (2004), 234–240. DOI:http://
dx.doi.org/10.1007/s00779-004-0283-x
Tiiu Koskela, Kaisa V¨
a¨
an¨
anen-Vainio-Mattila, and Lauri Lehti. 2004. Home is where your phone is: Us-
ability evaluation of mobile phone UI for a smart home. In Proceedings of International Conference on
Mobile Human-Computer Interaction (MobileHCI’04). Springer, 74–85. DOI:http://dx.doi.org/10.1007/
978-3-540-28637-0_7
Gerhard Leitner, Anton J. Fercher, and Christian Lassen. 2013. End users programming smart homes –
A case study on scenario programming. In Proceedings of the Workshop on Human-Computer Interac-
tion and Knowledge Discovery in Complex, Unstructured, Big Data (HCI-KDD’13). Springer, 217–236.
DOI:http://dx.doi.org/10.1007/978-3-642-39146-0_20
Loxone. 2016. Loxone Config. Accessed April 5, 2016 from http://www.loxone.com/enen/products/software/
loxone-config.html.
Michael McCurdy, Christopher Connors, Guy Pyrzak, Bob Kanefsky, and Alonso Vera. 2006. Breaking the
fidelity barrier: An examination of our current characterization of prototypes and an example of a mixed-
fidelity success. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
(CHI’06). ACM, New York, NY, 1233–1242. DOI:http://dx.doi.org/10.1145/1124772.1124959
Sarah Mennicken and Elaine M. Huang. 2012a. Hacking the natural habitat: An in-the-wild study of
smart homes, their development, and the people who live in them. In Proceedings of International
Conference on Pervasive Computing (Pervasive’12). Springer, 143–160. DOI:http://dx.doi.org/10.1007/
978-3-642-31205-2_10
Sarah Mennicken and Elaine M. Huang. 2012b. Why can’t I have both? The tension between comfort and
control in smart homes. In Proceedings of the Pervasive Intelligibility Workshop.
Sarah Mennicken, Jo Vermeulen, and Elaine M. Huang. 2014. From today’s augmented houses to tomorrow’s
smart homes: New directions for home automation research. In Proceedings of the 2014 ACM Interna-
tional Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’14). ACM, New Yo rk, N Y,
105–115. DOI:http://dx.doi.org/10.1145/2632048.2636076
myGEKKO. 2016. myGEKKO ConfiguConfig Interface. Accessed April 5, 2016 from http://www.my-gekko.
com/en/products/mygekko-app-software/.
Mark W. Newman. 2006. Now were cooking: Recipes for end-user service composition in the digital home. In
Proceedings of CHI’06 Workshop: IT@Home.
Mark W. Newman, Ame Elliott, and Trevor F. Smith. 2008. Providing an integrated user experience of
networked media, devices, and services through end-user composition. In Proceedings of International
Conference on Pervasive Computing (Pervasive’08). Springer, 213–227. DOI:http://dx.doi.org/10.1007/
978-3-540-79576-6_13
Ninja Blocks. 2016. Homepage. Accessed April 5, 2016 from http://ninjablocks.com/.
Jason Pascoe. 1997. The stick-e note architecture: Extending the interface beyond the user. In Proceedings
of the 2nd International Conference on Intelligent User Interfaces (IUI’97). ACM, 261–264. DOI:http://
dx.doi.org/10.1145/238218.238344
James Pierce and Eric Paulos. 2012. Beyond energy monitors: Interaction, energy, and emerging energy
systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM,
665–674.
Erika Shehan Poole, Marshini Chetty, Rebecca E. Grinter, and W. Keith Edwards. 2008. More than meets
the eye: Transforming the user experience of home network management. In Proceedings of the ACM
Conference on Designing Interactive Systems, DIS’08. ACM, 455–464. DOI:http://dx.doi.org/10.1145/
1394445.1394494
Michael Rietzler, Julia Greim, Marcel Walch, Florian Schaub, Bj¨
orn Wiedersheim, and Michael Weber.
2013. homeBLOX: Introducing process-driven home automation. In Proceedings of the 2013 ACM Con-
ference on Pervasive and Ubiquitous Computing Adjunct (UbiComp’13). ACM, 801–808. DOI:http://
dx.doi.org/10.1145/2494091.2497321
Tom A. Rodden, Joel E. Fischer, Nadia Pantidi, Khaled Bachour, and Stuart Moran. 2013. At home
with agents: Exploring attitudes towards future smart energy infrastructures. In Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems,CHI’13. ACM, New York, NY, 1173–
1182. DOI:http://dx.doi.org/10.1145/2470654.2466152
Jennifer A. Rode, Eleanor F. Toye, and Alan F. Blackwell. 2004. The fuzzy felt ethnography – Understanding
the programming patterns of domestic appliances. Pers. Ubiq. Comp. 8, 3–4 (2004), 161–176. DOI:http://
dx.doi.org/10.1007/s00779-004-0272-0
Jennifer A. Rode, Eleanor F. Toye, and Alan F. Blackwell. 2005. The domestic economy: A broader unit of
analysis for end user programming. In Proceedings of the CHI’05 Extended Abstracts on Human Factors
in Computing Systems. ACM, 1757–1760. DOI:http://dx.doi.org/10.1145/1056808.1057015
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.

Exploring End User Programming Needs In Home Automation 11:35
Shaun Salzberg. 2011. HomeMaestro. Retrieved from http://www.shaunsalzbergdesign.com/folio/home-
maestro/; http://shaunsalzberg.com/medialab/homemaestro
Leila Takayama, Caroline Pantofaru, David Robson, Bianca Soto, and Michael Barry. 2012. Making technol-
ogy homey: Finding sources of satisfaction and meaning in home automation. In Proceedings of the 2012
ACM Conference on Ubiquitous Computing. ACM, 511–520.
Maryam Tohidi, William Buxton, Ronald Baecker, and Abigail Sellen. 2006. User sketches: A quick, in-
expensive, and effective way to elicit more reflective user feedback. In Proceedings of the 4th Nordic
Conference on Human-computer Interaction: Changing Roles (NordiCHI’06). ACM, New York, NY, 105–
114. DOI:http://dx.doi.org/10.1145/1182475.1182487
Khai N. Truong, Elaine M. Huang, and Gregory D. Abowd. 2004. CAMP: A magnetic poetry inter-
face for end-user programming of capture applications for the home. In Proceedings of International
Conference on Ubiquitous Computing (UbiComp’04). Springer, 143–160. DOI:http://dx.doi.org/10.1007/
978-3-540-30119-6_9
Blase Ur, Melwyn Pak Yong Ho, Stephen Brawner, Jiyun Lee, Sarah Mennicken, Noah Picard, Diane Schulze,
and Michael L. Littman. 2016. Trigger-action programming in the wild: An analysis of 200,000 IFTTT
recipes. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM,
3227–3231.
Blase Ur, Jaeyeon Jung, and Stuart Schechter. 2013. The current state of access control for smart devices in
homes. In Proceedings of the Workshop on Home Usable Privacy and Security (HUPS’13).
Blase Ur, Elyse McManus, Melwyn Pak Yong Ho, and Michael L. Littman. 2014. Practical trigger-action pro-
gramming in the smart home. In Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI’14). ACM.
Marcel Walch, Michael Rietzler, Julia Greim, Florian Schaub, Bj¨
orn Wiedersheim, and Michael Weber.
2013. homeBLOX: Making home automation usable. In Proceedings of the 2013 ACM conference on
Pervasive and Ubiquitous Computing Adjunct (UbiComp’13) Adjunct Proceedings. ACM, 295–298.
DOI:http://dx.doi.org/10.1145/2494091.2494182
Rayoung Yang and Mark W. Newman. 2013. Learning from a learning thermostat: Lessons for intelligent
systems for the home. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and
Ubiquitous Computing. ACM, 93–102.
Miguel A. Zamora-Izquierdo, Jose Santa, and Antonio F. Gomez-Skarmeta. 2010. An integral and networked
home automation solution for indoor ambient intelligence. IEEE Perv. Comput. 9, 4 (2010), 66–77.
DOI:http://dx.doi.org/10.1109/MPRV.2010.20
Bin Zhang, Pei-Luen Patrick Rau, and Gavriel Salvendy. 2009. Design and evaluation of smart home user
interface: Effects of age, tasks and intelligence level. Behav. Inf. Technol. 28, 3 (2009), 239–249.
Received April 2016; revised January 2017; accepted January 2017
ACM Transactions on Computer-Human Interaction, Vol. 24, No. 2, Article 11, Publication date: April 2017.