Content uploaded by Georg Vogt
Author content
All content in this area was uploaded by Georg Vogt on Aug 19, 2016
Content may be subject to copyright.
Ina Renz 9-132-15
ICT instruments in multi-apartment buildings:
Efficiency and effects on energy consumption behaviour
Dr. Ina Renz, Institute for Housing and Environment (IWU)
Rheinstr. 65
64295 Darmstadt
Germany
i.renz@iwu.de
Georg Vogt (empirica)
Oxfordstr. 2
53111 Bonn
Germany
Georg.Vogt@empirica.com
Abstract
Article 9-11 of the energy efficiency directive [2012/27/EU] requires all EU members to introduce
individualised metering and billing to multi-apartment buildings. Inducing behavioural change in consumption is
a cost-efficient path to unleash the saving potentials in the building stock being only slowly renovated.
This paper describes the key results of the EU-funded ICT PSP project BECA (Balanced European Conservation
Approach) and analyses the effectiveness and efficiency of ICT feedback instruments in field trials that cover
heat energy, hot water consumption as well as electricity consumption and have been implemented at seven pilot
sites in seven countries including more than 1,500 pilot households in total.
Following a quasi-experimental design with pre-post comparisons and comparisons with control groups, the
effects of the services on behaviour change is investigated. Evaluation of effectiveness of services is carried out
on two levels: (1) the level of energy consumption and (2) the level of specific energy behaviour based on panel
survey data allowing for comparisons of individual behaviour before and after service operation. Furthermore, a
multi-stakeholder cost-benefit analysis widely following EC guidance [2012/148/EU] compares the ‘Do-
Nothing’ scenario with the cost and benefits of BECA ‘Intervention’.
The trials provide evidence that ICT feedback instruments focusing on behaviour can change domestic energy
behaviour and lead to energy savings. Energy behaviour was mainly affected with respect to heating and
electricity whilst there was found no meaningful influence on specific behaviour related to hot water
consumption investigated at BECA. The success of services is furthermore influenced by service and user-
related factors as well as local circumstances such as combination with management services applying on
building level, motivational aspects or financial conditions. Socio-economic break even is reached in all pilot
sites.
Introduction
Making domestic energy use more visible to the consumers, feedback instruments present a strategy to induce
energy saving behaviour in households. Although first studies in the 1970s already found that feedback – mostly
provided via display monitors – has measurable effects and is worth advancing (Darby 2006), the development
of feedback instruments and their evaluation is only gradually based on knowledge from the social sciences.
Hence the evaluation of both attitude/behaviour change and achieved consumption reduction is either missing,
does not follow (common) methodological standards or is based on small and short-term studies (Farley / Mazur-
Stommen 2014; Ehrhard-Martinez et al. 2010; Abrahamse 2007). Furthermore, Pierce et al. (2010) present a
more critical view on residential feedback instruments and raise questions about their limitations, the non-
negotiability of domestic interactions resulting in an unwillingness to change these interactions and about
alternatives that might present simpler ways to achieve the same consumption reduction. Current studies focused
its research mainly on three different questions: (1) theory-oriented research with no deep interest in impact
assessment of instruments or practice transfer (Fischer 2008), (2) application-oriented studies often realised by
energy suppliers or housing providers who are not accompanied by researchers (Farley / Mazur-Stommen 2014;
Abrahamse 2007) or (3) design-oriented computer science projects centring on tool development (Froehlich et
al. 2010, Weiss et al. 2012). Furthermore, published feedback studies often emphasise electricity consumption
neglecting heat energy or water consumption. Although multi-apartment buildings seem to be promising objects
for offering and examining feedback services, the effectiveness of theory-based feedback techniques has not yet
been sufficiently demonstrated in this context (Farley / Mazur-Stommen 2014).
This paper contributes to close the gap between these three research approaches and presents scientifically
substantiated findings about the effectiveness and efficiency of the services across the pilot sites with a focus on
RUAS representing the behavioural service of BECA (Balanced European Conservation Approach). It addresses
the questions whether BECA’s behavioural service has an impact on energy / resource consumption with regard
to heat energy, domestic electricity as well as hot water consumption and on specific energy behaviour related to
those consumption types. It also questions whether such services are cost effective.
BECA involved experts from all relevant disciplines in order to develop adequate ICT instruments for improving
energy efficiency in private households as well as to ensure their evaluation by an independent scientific
organisation. The project developed and tested services targeting the optimisation of user behaviour and the
increase of energy and resource awareness (RUAS – Resource User Awareness Service) and addressing energy
management issues (RMS – Resource Management Service). The project equipped 2,300 social housing
dwellings with metering equipment for the monitoring of their energy and resource consumption in seven pilot
sites located in seven European countries. The services have been tested under real conditions in approximately
1,500 dwellings. The remaining dwellings were established as control group where smart metering equipment
was installed, but no services were provided. The particular characteristic of RUAS is the combination of
insights from behaviour sciences and environmental psychology with knowledge of technology developers and
ICT specialists.
The paper describes the theoretical and methodological approach presenting the design of the field trials having
been implemented in the pilot sites and data as well as variables used for analyses. Subsequently the findings
about the effectiveness of services regarding to energy / resource consumption and particular energy behaviour
are presented. Afterwards the results of a cost-benefit-analysis are documented which address the efficiency of
services. The concluding section summarises and discusses the main findings and also indicates further research
needs.
Theoretical approach and design of the field trials at pilot sites
BECA combines knowledge from different scientific disciplines and practical experiences implementing ICT-
solutions to change behaviour (RUAS), the topic of this paper, and automated resource management (RMS)
using the same infrastructure. RUAS applies ICT feedback instruments in order to influence attitudes and
knowledge of users and especially to encourage them to change their behaviour to energy saving manners. The
development of these feedback services are oriented at theories of behaviour change (Schwartz et al. 2014),
especially a rational-economic approach and, in particular, the Theory of Planned Behaviour (TPB; Ajzen 1991).
The rational-economic approach assumes that people act to maximise monetary rewards and minimise costs,
whereas the TPB also implies an expectancy-value construction meaning that people “behave according to their
beliefs about the outcomes of their behaviour and the values they attach to those outcomes” (Jackson 2005: 46).
These beliefs are not restricted to financial aspects, but also include attitudinal, normative or affective aspects,
control beliefs as well as contextual factors. In doing so, it opens the straitened vision of economic approaches
focussing on monetary outcomes to benefits in terms of effort and social approval (Jackson 2005, Steg/Vlek
2009). The RUAS service is based on two concepts of behaviour science: (1) cognition meaning that information
is given to a particular audience including education and training features; (2) calculus, which is the monetary
benefit for users resulting from the achieved savings (Farley / Mazur-Stommen 2014). Furthermore, RUAS
implemented several motivation techniques that are going beyond the exclusive provision of energy
consumption feedback such as experience exchange in focus groups during the service development and directly
involving user experiences and opinions by applying a two-iteration development approach (Geller et al 1990:
130). This approach established a mutual process between technology development and user practices whereby
end users are more likely to engage with the services provided (and thus benefit from them), because their
perspective is seriously taken into account (Rohracher/Ornetzeder 2006; Steg/Vlek 2009). In general, features
included comparative feedback (comparison with similar dwellings located at the same pilot site) and historic
feedback mostly with possibilities to chose different time periods for comparison (e.g. specific month, previous
year). Information was displayed using simplified graphics and charts including an alert or traffic light system.
1
In addition, energy saving tips have been provided and contact points for clarification of questions have been
established. Feedback has been provided on a monthly basis. Tenants without internet access at home were able
to use public internet points established by the housing providers. Furthermore, monthly paper reports as well as
educational material have been provided at most pilot sites.
1
For screenshots of the portals see Renz et al. 2014, Vogt et al. 2012 or
http://source.smartspaces.eu/80_References/BECA/ (last access: February 2015).
The RMS service comprises the implementation of a monitoring system in order to ensure an error-free
operation of the technical infrastructure and give maintenance warnings to the pilot site staff. At some pilot sites,
RMS includes automated features for optimising the heating system (e.g. by taking into account outside
temperatures) or setting a limit for indoor temperature.
2
In summary, RMS aims at the optimisation of the
technical systems affecting an entire building and the monitoring of consumption data that was used for RUAS
aiming at the optimisation of the tenants’ energy behaviour.
As part of the project, all participating dwellings of experimental and control groups have been equipped with
smart metering equipment for the monitoring of their energy and resource consumption at the beginning of the
project. In both evaluation groups consumption data have been measured covering 12 month before and 12
month after service implementation.
3
The following table provides an overview of pilot site trials aiming to affect users’ energy behaviour. This
includes the provision with (1) RUAS or (2) RUAS being provided in combination with RMS. Dwellings
provided with RMS alone are not listed as they are not part of this analysis. The dwelling numbers in
experimental and control groups are displayed after data cleansing
4
of consumption data.
Table 1: Pilot sites with service types and numbers of dwellings with consumption data available in
comparison groups
Pilots
Service type
Consumption type
Experimental group
Control group
Belgrade1
Combined RUAS+RMS
Heating
46
92
RUAS
Cold water
92
92
Electricity
79
69
Darmstadt
Combined RUAS+RMS
Heating
210
139
RUAS
Cold water
189
368
Hot water
188
369
Havirov
RUAS
Heating
63
n/a
Cold water
28
n/a
Hot water
30
n/a
Manresa
Combined RUAS+RMS
Heating
23
43
RUAS
Cold water
27
41
Electricity
28
41
Örebro
Combined RUAS+RMS
Cold water
67
n/a2
Hot water
67
n/a2
Ruse
Combined RUAS+RMS
Cold water
26
35
Electricity
32
41
Torino
Combined RUAS+RMS
Heating
39
44
Hot water
39
43
RUAS
Heating+Hot water
149
179
Heating
55
31
Cold water
219
161
1 At Belgrade consumption data for heating and cold water are measured building-wise and are allocated to all
dwellings in equal measure based on their surface area respectively number of persons living in the dwelling.
Therefore consumption data for heating and cold water is excluded from the analyses.
2 At Örebro a control group was originally planned, but could not be realised because the web portal was
implemented as additional part of the general website of the housing provider that all tenants could access
through their existing passwords. Consequently, these households were excluded from the analyses.
Methodological Approach for impact assessment
The evaluation approach follows a quasi-experimental design meaning that dwellings / households have not been
randomly assigned to experimental or control group, but housing providers decided about the group assignment
ensuring that buildings and households of both groups show similar characteristics (e.g. building type, energy
performance of buildings, tenant characteristics). Tenants of the control group have not been informed about the
2
For further information see Vogt et al. (2012).
3
For further information see section “Methodological approach”.
4
By data cleansing dwellings with a change of tenancy or implausible respectively missing values due to service
malfunctions were excluded from analysis.
services, tenants of the experimental group have been informed and invited to participate in the refinement of the
portals. The effectiveness of the behavioural service (RUAS) was assessed by applying pre-post and control
group comparisons on the dwelling, respectively household level. This approach allows the identification of the
net feedback effect and justifies a causal interpretation of the estimates. Hence, savings or behaviour changes of
the experimental group can be interpreted as solely caused by the services and not by programme-external
factors.
Analyses are carried out on two levels: (1) The level of energy consumption related to heating (kWh), hot water
consumption (cbm) and electricity (kWh) and (2) the level of particular energy behaviour related to those
consumption types.
Analyses on consumption level are based on measured dwellingwise consumption data covering 12 months
before service provision (baseline period: November 2011- October 2012) and 12 months after starting the field
trials (reporting period: November 2012 - October 2013).
Analyses about the energy behaviour (level 2) are based on survey panel data with a two-stage data collection.
The first stage was realised briefly before service provision in autumn 2012 (baseline survey), the second stage
was realised at the end of the reporting period in autumn 2013 (final survey). Surveys were programmed with
help of a software tool allowing for computer assisted personal surveys or using the printed questionnaires as
postal surveys.
5
The response rates averaged at 25-36% in the specific survey stages which is adequate in view
of social housing tenants. In the panel sample are included 184 tenants that belong to the experimental group that
received feedback services solely ore in combination with RMS and 149 tenants belonging to the control group.
An overview on survey behaviour statements is given in Table 4. For the analyses, only households which
participated at both survey stages are considered.
In this paper two experimental groups representing different treatment types have been distinguished:
1. Experimental group combined services: In order to base analyses on high sample sizes the service types
RUAS and combined RUAS+RMS have been consolidated in one experimental group. This group applies to
all consumption types.
2. Experimental group RUAS: Dwellings in this group received RUAS alone. This group applies to heating
and electricity where RMS included automated features at some pilot sites or other restrictions are given that
are unrelated to the tenants’ behaviour.
6
Dependent variables
At the energy consumption level (1), the achieved annual energy consumption savings of dwellings (difference
of consumption in reporting and baseline period) are used as dependent variables. The saving calculations are
based on adjusted consumption values. For all consumption types the dwelling surface has been taken into
account, heating consumption values additionally have been Heating Degree Day (HDD) corrected taking into
account the outside temperature (Kelvin days per sqm). We interpret (higher) energy savings of the experimental
group against the control group as a result from changed behaviour due to the feedback service.
At the energy behaviour level (2), a set of specific energy behaviour related to heating, hot water and electricity
consumption is in depth analysed. Each consumption type has been surveyed by several behaviour statements
(for an overview see Table 4) as 5 level Likert question.
7
Dependent variables indicate the individual change of
behaviour for every statement. They have been constructed as binary variable indicating whether ones energy
behaviour was improved or not. The variable was calculated as difference of the tenants’ agreement rates at the
baseline and final survey (taking values between -4 and 4) that was dichotomised in a second step. Values >0 are
5
When conducting postal surveys, pilot sites collected the questionnaires and entered the answers using the
questionnaire programmed at their computers. Data sets could be automatically exported and were then sent to
the evaluation institution.
6
Heating RMS at some pilots includes automated features optimising the heating system that apply on building
level and are not related with the tenants’ consumption behaviour. The pilot site providing combined services for
electricity also uses electricity for heating. Therefore changes of electricity consumption behaviour might be less
obvious.
7
Answer categories represent levels of agreement for each statement (strongly agree, rather agree, neither
agree nor disagree, rather disagree, stongly disagree) with additional categories dont’t know and not
appropriate. Two more statement have been surveyed, but are not inclueded in the analysis: „My room
temperature at night usually is lower than by day“, because it is not known for all dwellings whether an
automatic night setback is available. „I mostly tumble dry my clothes“, because the sample sizes are too small
due to low numbers of tenants possessing a dryer.
coded as improvement and values <=0 are coded as no improvement (including worsening and no change).
8
Dichotomisation of variables is a common strategy in the social sciences which is used if numbers in single
categories are rather low. Consequently, more appropriate analyses can be done applying logistic regression
which is a statistical procedure for binary dependent variables.
Independent variables
In the multivariate analyses of particular behaviour a set of independent variables is included where appropriate.
These variables are assumed to be programme-external factors affecting energy behaviour. They serve as control
variables in the regression models, controlling for differences between experimental group and control group
with regard to theses aspects. User-related variables include the tenants’ interest in saving energy at home and
the energy saving norm following the statement “I think I should save energy at home”
9
that have been taken
from the baseline survey.
10
The age of tenants is also included where appropriate following three age groups: up
to 41 years, 42-59 years, >59 years. Local circumstances include the availability of financial support for rent or
service charges by the municipality or other public institutions and the availability of consumption related billing
of energy consumption.
11
As initial situation the baseline energy consumption was included for estimates of
energy savings and the reported baseline behaviour (dummy variable representing whether behaviour already
corresponds to the recommended behaviour or not) was included for estimates of behaviour change.
Results of the pilot trials
We first present the results of RUAS effectiveness on energy resource consumption, starting with some
descriptive results followed by OLS estimates. Then results from the effects of RUAS on particular energy
behaviour are presented. In the third part findings of the cost-benefit analysis are documented.
Effects of services on energy/resource consumption
Results of descriptive analyses
Table 2 shows figures for the annual mean consumptions during baseline and reporting period (adjusted values)
and the average of savings respectively increased consumptions based on the dwellingwise difference of
reporting and baseline consumption. Negative figures display savings, positive figures increased consumptions.
With respect to heating, both the experimental and control groups saved heat energy based on the adjusted values
taking into account outside temperatures and surface of the dwelling. Thereby both treatment groups save
somewhat more energy than the control group whereby the combined services group achieved the highest
savings. But those savings cannot be interpreted as solely caused by behaviour change because automatic
systems regulations are included there (as described above). In contrast, the RUAS group shows no savings on
the level of averaged individual percentage savings. The relative low savings and modest influence of the
services might be explained by the fact that both treatment groups during the baseline stage already spent less
heat energy than the control group did. Therefore the potential for further savings or more precisely said the
pressure and motivation to save energy and money might have been lower in those groups than in the control
group.
With respect to hot water consumption no positive influence of the services can be found if the average savings
in kWh/sqm are taken into account. However, the averaged individual percentage savings hint at a positive
influence of the treatment group that received the combined services. This can be explained by some households
with comparatively high percentage savings against households with more moderate increased consumptions.
8
Cases with unchanged behaviour not having any potential for improvement (answer category strongly agree at
baseline survey) have been excluded, because this would not be a benefit from the services.
9
Both statements have been surveyed as 5-level Likert questions (agreement scale with same categories than
behaviour statements) and have been dichotomised for analysis (strongly and rather agree = characteristic
available, further categories = characteristic not available).
10
The question whether those motivational factors also have been influenced by the services is not investigated
in this paper which is focussed on the effects on actual behaviour and consumption, but was discussed on the
pilot site level in the final report (see Renz et al. 2014). Taking the information from the baseline survey ensures
that motivational factors are programme-external and therefore not influenced by the services.
11
Further aspects such as income or education level could not be included due to low sample sizes (high item
non response).
The results for electricity show that all groups have achieved savings, but savings of the RUAS treatment group
are most striking. This might be partly due to the fact that households of the RUAS group spent most energy for
electricity during the baseline period and therefore they had higher potential for achieving savings.
Table 2: Average consumption in baseline and reporting period and average savings per consumption
type and treatment type
Consumption
type
(Unit/a)
Evaluation group
N
Baseline
Reporting
Average savings /
incr. consumption
Mean
Std. dev.
Mean
Std. dev.
Unit
% per dwelling
Heating
(Kelvin
days/sqm)
Contr. gr.
134
0.075
0.045
0.085
0.043
-0.002
-0.1
Exp. gr. comb. services
130
0.053
0.037
0.047
0.029
-0.006
-5.6
Exp. gr. RUAS
73
0.051
0.036
0.048
0.033
-0.003
1.0
Hot water
(cbm per
sqm)
Contr. gr.
81
0.35
0.26
0.33
0.25
-0.02
-0.5
Exp. gr. comb. services
138
1.59
3.36
1.74
4.26
0.15
-7.8
Exp. gr. RUAS
55
3.29
4.85
3.91
6.17
0.09
5.4
Electricity
(kWh per
sqm)
Contr. gr.
94
55.89
41.99
52.44
41.54
-3.45
-2.0
Exp. gr. comb. services
83
57.37
38.42
55.97
38.99
-1.40
1.0
Exp. gr. RUAS
10
104.35
53.20
91.43
57.56
-12.92
-12.7
Results of multivariate analyses
The following multivariate analyses will shed some light on these assumptions. Table 3 shows the results of OLS
regressions displaying the parameter estimates which represent the change in the outcome associated with a unit
change in the predictor. Therefore parameters for combined services respectively RUAS show the change of
savings if the household belongs to the experimental group instead to the control group. Models were calculated
for both treatment types where appropriate and control for the baseline consumption. There can be found hints
for positive influences of the services for all consumption types. With respect to heating a significant impact is
found for the influence of the combined services. But, as discussed above, this cannot be interpreted as being
solely caused by behaviour change because automatic heating regulation services are partly included in the
combined services. However, slight influence can also be found for the RUAS treatment.
The impact for services related to hot water consumption is more striking where households of the combined
services group achieve by 0.13 cbm/sqm higher savings than the control group.
With respect to electricity a remarkable impact can be found for the RUAS treatment where households save
about 5.4 kWh/sqm more electricity than the control group. The result of the combined services group that hints
at an increased consumption of this group is partly due to the fact that electricity in the pilot site offering
combined services is also used for heating.
Table 3: Impact of services on energy/resources consumption (results of OLS regression)
Variable
Heating
(Kelvin days per sqm)
Hot Water
(cbm/sqm)
Electricity (kWh/sqm)
M1
M2
M1
M1
M2
Combined Services (dummy)
-0.008
(0.002)
***
-0.128
(0.091)
2.176
(2.432)
RUAS (dummy)
-0.003
(0.002)
-5.399
(6.177)
Baseline consumption
(adjusted)
-0.170
(0.024)
***
-0.094
(0.023)
***
0.246
(0.016)
***
-0.083
(0.030)
***
-7.631
(2.210)
***
Constant
0.011
(0.002)
***
0.005
(0.002)
*
-0.108
(0.071)
1.169
(2.372)
1.239
(2.901)
R2 (adjusted)
0.17
0.07
0.52
0.03
0.05
N
264
207
219
177
104
Note: * indicates significance at p < 0.1, ** at p < 0.05 and *** at p < 0.01.
All models besides the model for hot water consumption show rather modest values for adjusted R2 meaning that
there are further variables having an influence on the achievement of savings and might increase the quality of
the models. Unfortunately, due to restrictions of the samples
12
, it is not useful to include information basing on
12
In many cases with panel survey data available, consumption data is missing due to malfunctions of the
metering devices or – in the case of electricity – due to cut-offs as a consequence of outstanding costs.
the survey in the models.
13
However, the findings suggest that the feedback services have at least moderate
impacts on the energy behaviour for all three consumption types when controlling for the baseline consumption.
Impact of services on energy behaviour
In this section the impact of the feedback services on particular energy behaviour is investigated in more detail.
Results of descriptive analyses
Table 4 shows the proportions of tenants with improved behaviour and displays the sizes for the panel samples in
each group. With respect to heating the mostly higher proportions for improved behaviour
14
in both treatment
groups against the control group suggests a positive influence of the services. Only for keeping shut windows
and doors for commonly used rooms control group tenants performed better than tenants of the experimental
groups. The behaviour statements related to hot water do not hint at a positive influence of the services as
proportions for improvement in control groups are the highest for all statements. With respect to electricity
energy behaviour, tenants that receive the combined services show higher improvement scores for the majority
of statements whereas RUAS services display heterogeneous results.
Table 4: Behaviour change in tenant households
Consump.
type
Energy behaviour statements
Improved behaviour (%)
N in treatment group
Contr.
gr.
Combined
services
RUAS
Contr.
gr.
Combined
services
RUAS
Heating
I turn off the heating when I open the windows
32.9
50.8
68.6
85
61
35
I turn the heating down when I leave a room
unused
23.5
24.1
32.0
81
54
25
I turn the heating down when I leave my home for a
longer time
30.2
34.3
40.9
43
35
22
In winter time: I mind to keep shut windows and
doors for commonly used rooms
32.1
17.5
24.1
84
40
29
Hot water
I rather take a shower instead of a bath
44.7
30.3
n/a
38
33
n/a
I use cold water to wash my hands
32.7
28.2
n/a
104
78
n/a
I wait until I have a full load before I use my
washing machine or dishwasher
48.1
37.8
n/a
52
37
n/a
Electricity
I turn out the light when no one is in the room
22.2
24.1
44.4
54
29
9
I switch off TV or other equipment when there is no
one in the room for a longer time
39.1
31.0
0.0
69
29
5
I completely switch off an appliance with Stand by-
function when I have finished using it
30.2
34.0
22.2
106
53
9
I unplug chargers from the mains
38.5
44.1
50.0
65
34
10
I mind the energy consumption when I purchase
new electric appliances
25.3
33.3
20.0
87
39
5
I wait until I have a full load before I use my
washing machine or dishwasher
48.1
34.5
37.5
52
29
8
Results of multivariate analyses
The subsequent logistic regression analyses control for further influences, in order to draw conclusions on the
net impact of the services. Generally, including the control variables in the models, remarkably contributed to
increase the model fit. Results present the odds ratios for improved behaviour (against not improved behaviour)
for the single behaviour statements if the particular experimental groups are compared to the control group.
There is displayed the effect-coefficient Exp(b) which shows the effect sizes indicating the factor by which the
chance for an improved behaviour (y=1 improved against y=0 not improved) changes when the independent
variable changes by one unit (metric variables) or if the observed category of the independent variable is
compared with the reference category. Consequently, the chance for improved behaviour increases if exb(B) > 1,
it decreases if exp(B) < 1 and it doesn’t change if exp(B) = 1. Therefore exb(B) > 1 indicates a positive net
impact of the service whereby the larger exb(B), the stronger the influence. Standard errors are displayed in
parentheses.
13
The same is true for the inclusion of reported behaviour in consumption models.
14
As described above, improved behaviour as binary variable (improved vs. not improved) represents the
dependent variable for behaviour analyses.
Heating
For heating, the results show that behaviour is optimised especially by households where only RUAS is provided
(Table 5, Models 2). In this case positive impacts can be found for three of the four statements. Through the
provision of combined services households improved their behaviour related to two statements. For both
treatment types the biggest impact can be found for turning off the heating when opening the windows. For this
kind of behaviour the chance for an improved behaviour (against control group) increases by 2.19 if the
household receives the combined services and even increases by 3.32 if the household receives RUAS only. In
other words the chance that the tenants optimised their behaviour is more than twice as high if they receive the
combined services and is more than three times as high if they receive RUAS.
RUAS also shows a positive, but lower influence on turning the heating down when leaving a room unused and
turning the heating down when leaving the home for a longer time whereas the combined services indicate no
remarkable effect. The behaviour to keep shut windows and doors for commonly used rooms in winter time is
not improved by the services. All at once this is the model with the worst model fit whereas the other models
show high values with an explained variance of more than 40% displayed by Nagelgerkes R2 and correctly
predicted cases about 80% for nearly all models and service types. However, the treatment estimates are not
statistically significant which should be mainly due to the rather low sample sizes and should not be over
interpreted.
15
The higher impact of RUAS in comparison to the combined services might be explained by rebound effects
within the combined services group. For example, tenants might be less motivated to change their behaviour if
they know that an automatic regulation of the heating system will already lead to lower heat energy
consumption. However, this assumption cannot be further examined.
Table 5: Odds ratios for improved behaviour related to heating consumption
(results of logistic regressions)
Variable
I turn off the heating
when I open the
windows
I turn the heating down
when I leave a room
unused
I turn the heating down
when I leave my home
for a longer time
In winter time: I mind to
keep shut windows and
doors for commonly
used rooms
M1
M2
M1
M2
M1
M2
M1
M2
Combined services
(dummy)
2.190
(0.639)
0.729
(0.688)
1.125
(0.841)
0.644
(0.879)
RUAS
(dummy)
3.315
(0.746)
1.469
(0.893)
1.769
(0.902)
0.862
(0.904)
Initially recomm.
behaviour (dummy)
0.026
(0.615)
***
0.038
(0.618)
***
0.067
(0.627)
***
0.070
(0.710)
***
0.049
(0.832)
0.060
(0.880)
***
0.043
(0.837)
***
0.029
(0.947)
***
Interest in saving
energy (dummy)
2.254
(0.956)
2.318
(0.954)
0.312
(1.057)
0.414
(1.030)
0.516
(1.132)
0.968
(1.271)
3.206
(1.166)
3.065
(1.189)
Energy saving norm
(dummy)
1.034
(0.598)
0.904
(0.654)
0.791
(0.720)
0.514
(0.777)
0.329
(0.901)
0.21
(0.983)
0.379
(0.886)
0.251
(0.964)
Financial support
(dummy)
2.983
(0.942)
1.815
(0.967)
1.106
(0.856)
1.060
(1.012)
0.947
(1.209)
1.606
(1.294)
0.859
(1.359)
0.694
(1.395)
Consumption related
billing (dummy)
0.537
(0.683)
0.696
(0.742)
0.303
(0.788)
0.596
(0.879)
0.904
(0.920)
1.033
(0.973)
0.292
(0.927)
0.197
(0.999)
Age (up to 41 years
= ref.)
42-59 years
1.155
(0.685)
0.542
(0.701)
0.889
(0.826)
0.949
(0.812)
8.135
(1.106)
*
0.529
(0.948)
0.900
(0.814)
0.972
(0.656)
More than 59 years
0.480
(0.715)
0.958
(0.713)
0.759
(0.865)
0.621
(0.953)
4.687
(1.186)
0.100
(1.241)
*
0.811
(0.834)
1.734
(0.656)
N
112
95
94
75
61
53
98
90
Nagelkerkes R2
0.569
0.529
0.424
0.415
0.498
0.462
0.347
0.369
Correctly predicted
cases (%)
82.1
81.1
77.7
74.7
83.6
77.4
75.5
77.8
Note: * indicates significance at p < 0.1, ** at p < 0.05 and *** at p < 0.01.
15
It also should be taken into account that the generalisation of the results is limited due to restriction of the
sample on social housing.
One more interesting aspect is the question whether treatment effects vary for specific subgroups of tenant
households. Regression models including interaction terms can shed some light on this question. Sample sizes
allowed comprehensive interaction models for turning off the heating when opening the windows and at least
one model for turning the heating down when leaving a room unused. The following figure shows the estimates
for the treatment main term of each interaction model
16
.
Whereas the odds ratios for the interaction of treatment and interest in saving energy at home do not vary
between tenants with and without interest and do not differ much from the odds in the main model (Table 5), the
other interactions related to turning off the heating when opening windows show different treatment effects
within the particular subgroups. Among tenants without financial support for rent or service charges and among
tenants without consumption related billing system, the effect of RUAS is much larger than for the subgroups
which have those features. The fact that tenants without financial support are more often improving their
behaviour suggests that feedback services might be more useful for target groups not belonging to social housing
and therefore not being financially supported.
Interestingly for tenants with a consumption related billing system RUAS has no positive effect at all. This might
be due to the fact that the tenants with consumption related billing already had been more sensitive for their
behaviour before receiving the services and therefore have been less responding to them. But further interaction
effects, e.g. between financial support and billing system might also be an explanation.
With respect to the age subgroups, RUAS shows a very high effect for the oldest group considered in the sample
(> 59 years) which also is statistically significant. Furthermore, RUAS also positively affects tenants below 42
years, whereas in the middle age group no positive influence can be found on that behaviour.
The only interaction term considered for turning the heating down when leaving a room unused shows no
remarkable differences for feedback effects in within both subgroups.
Note: * indicates significance at p < 0.1, ** at p < 0.05 and *** at p < 0.01.
Hot water
In the main effect models the behaviour related to hot water is not considerably influenced by the services (Table
6). Only using cold water for washing hands is somewhat more often optimised if the household receives the
services than in the control group. The estimates are not statistically significant, but quite robust standard errors
appear. The somewhat lower R2 of 33.9% for two behaviour statements indicate that there might be further
influencing variables that are not considered in the model. However, the available models do not indicate an
impact of the services on hot water energy behaviour. This means that the effect of the services on hot water
consumption that was found in the OLS model (s. Table 3) might be caused by further kinds of behaviour not
being asked in the survey.
However, the interaction models for hot water behaviour produce some interesting findings. The following
figure shows a big impact of the services on the improvement of taking a shower instead of a bath for tenants
who had not been interested in saving energy at home in the year before service implementation. This means that
the services succeeded to raise their interest and to motivate them for behaviour improvement. The same pattern
is found for the improvement of not using the washing machine or dishwasher before there is a full load. For
using cold water for washing hands, differences in subgroups are less obvious. But interestingly tenants without
16
In each interaction model only one interaction term is included in order to increase interpretability and model
fit.
3.34
3.08
1.18
3.64
0.90
11.45*
5.01
0.68
24.63**
2.11
1.55
0
5
10
15
20
25
30
available
not available
available
not available
available
not available
< 42 years
42-59 years
> 59 years
Interest in saving
energy at home
Financial support
Consum ption
related billing
Age
HEATING: ODDS RATIOS OF RUAS GROUP AGAINST CONTROL GROUP
IN SUBGROUPS
I turn off the heating
when I open the
windows
I turn the heating
down when I leave a
room unused
energy saving norm (statement: I think I should save energy at home) in the year before services implementation
show higher odds in the experimental group than in the control group than tenants already being aware that they
should save energy. Again the services succeeded to convince a group of tenants that not cared very much about
the topic before.
Table 6: Odds ratios for improved behaviour related to hot water consumption
(results of logistic regressions)
Variable
I rather take a
shower instead of a
bath
I use cold water to
wash my hands
I wait until I have a full
load before I use my
washing machine or
dishwasher
M1
M1
M1
Combined services
(dummy)
1.007
(0.744)
1.278
(0.485)
0.244
(0.983)
RUAS
(dummy)
Initially recommended
behaviour (dummy)
0.116
(0.680)
***
0.057
(0.569)
***
0.008
(1.185)
***
Interest in saving energy at
home (dummy)
0.219
(0.930)
1.458
(0.643)
0.755
(1.076)
Energy saving norm (dummy)
0.341
(0.783)
1.659
(0.449)
0.087
(0.825)
***
Financial support (dummy)
1.078
(0.941)
1.110
(0.634)
3.743
(1.114)
Consumption related billing
(dummy)
1.546
(0.971)
0.468
(0.608)
10.593
(1.255)
N
59
158
72
Nagelkerkes R2
0.339
0.339
0.627
Correctly predicted cases %
72.9
73.4
83.3
Note: * indicates significance at p < 0.1, ** at p < 0.05 and *** at p < 0.01.
Note: * indicates significance at p < 0.1, ** at p < 0.05 and *** at p < 0.01.
Electricity
With respect to electricity consumption the odds ratios vary largely by service type and between kinds of
behaviour, but however indicate positive influences in all cases. Whereas the impact of combined services
generally is rather moderate and odds ratios are not higher than 1.8, the impact of RUAS is striking. Households
that where provided by RUAS show nearly three times higher chances to improve their behaviour with respect to
completely switch off appliances with stand by-function and show nearly 5 time higher chances to unplug
chargers from the mains than the control group. The biggest impact can be found for turning out the light when
no one is in the room where the RUAS tenants optimised their behaviour by the factor 9.7 against the control
group. The last two effects are even statistically significant at p < 0.1. The high R2 and large proportions of
correctly predicted cases suggest a convincing model fit for all cases.
The more modest impact of combined services can be explained by the fact that the only pilot site that provided
combined services uses electricity not only for appliances, but also for heating.
0.65
27.38
1.28
1.30
0.81
3.09
2.97
1.10
0.16*
11.34
0
5
10
15
20
25
30
available
not available
available
not available
available
not available
Interest in saving energy
at home
Energy saving norm
Financial support
HOT WATER: ODDS RATIOS OF COMBINED SERVICES GROUP AGAINST
CONTROL GROUP IN SUBGROUPS
I rather take a shower
instead of a bath
I use cold water to wash my
hands
I wait until I have a full load
before I use my washing
machine or dishwasher
Table 7: Odds ratios for improved behaviour related to electricity consumption
(results of logistic regressions)
Variable
I turn out the light when
no one is in the room
I completely switch off
an appliance with Stand
by-function when I have
finished using it
I unplug chargers from
the mains
I switch off
TV or other
equipment
when there
is no one
in the room
for a longer
time1
I mind the
energy
consump-
tion when I
purchase
new elec-
tric appli-
ances1
M1
M2
M1
M2
M1
M2
M1
M1
Combined services
(dummy)
1.100
(0.664)
1.650
(0.476)
1.793
(0.620)
1.164
(0.667)
1.559
(0.528)
RUAS
(dummy)
9.736
(0.905)
**
2.697
(0.954)
4.953*
(0.885)
Initially recomm.
behaviour (dummy)
0.125
(0.758)
***
0.040
(0.936)
***
0.046
(0.513)
***
0.051
(0.611)
***
0.032
(0.747)
***
0.008
(1.233)
***
0.012
(0.891)
***
0.064
(0.540)
***
Interest in saving
energy (dummy)
0.543
(1.083)
0.079
(1.455)
*
0.119
(0.812)
***
0.112
(0.880)
**
0.055
(1.367)
**
0.028
(1.758)
**
0.153
(0.931)
**
2.333
(1.003)
Energy saving
norm (dummy)
2.836
(1.039)
2.261
(1.288)
2.125
(0.529)
1.826
(0.620)
0.646
(0.701)
1.240
(1.004)
1.433
(0.723)
1.412
(0.582)
Financial support
(dummy)
9.136
(1.342)
*
28.793
(1.881)
*
1.251
(0.626)
0.770
(0.755)
0.219
(1.109)
0.097
(1.504)
0.132
(1.278)
0.746
(0.738)
N
80
62
150
109
91
69
94
119
Nagelkerkes R2
0.185
0.415
0.448
0.415
0.546
0.648
0.586
0.353
Correctly predicted
cases %
82.5
82.3
79.3
81.7
83.5
87.0
86.2
82.4
Note: * indicates significance at p < 0.1, ** at p < 0.05 and *** at p < 0.01.
1 For those statements the impact for RUAS services could not be investigated due to low sample sizes.
Due to low sample sizes interaction models are only useful between energy saving norm and evaluation group
for two kinds of behaviour: to completely switch off appliances with Stand by-function and to unplug chargers
from the mains whereby divergent results can be found. For completely switching off the Stand by-function a
pattern similar to the other consumption types can be found: Tenants without energy saving norm are much more
likely to improve their behaviour if they belong to the RUAS group than tenants already showing this feature
during baseline. For unplugging chargers from the mains, inverse results occur. Maybe even if tenants already
had the impression that they should save energy, they did not know that unplugging chargers is one possibility to
do so and just have been learning this with help of the services. However, this assumption cannot be further
examined and does not explain why the services in this case did not succeed to motivate more tenants without
energy saving norm, too.
Note: * indicates significance at p < 0.1, ** at p < 0.05 and *** at p < 0.01.
Cost-Benefit Analysis
The Cost-Benefit Analysis (CBA), following EC recommendations for smart metering (EC 2012), compares the
'Do-Nothing' scenario with the ‘Intervention’ measures implemented in BECA (Vogt et. al. 2013). The CBA
collects a wide range of indicators in the areas implementation (CAPEX) such as metering and IT equipment,
1.80
8.14
6.32*
1.11
0
2
4
6
8
10
available
not available
Energy saving norm
ELECTRICITY: ODDS RATIOS OF RUAS GROUP AGAINST CONTROL
GROUP IN SUBGROUPS
I completely switch off
an appliance with Stand
by-function when I have
finished using it
I unplug chargers from
the mains
operation (OPEX) such as provision of energy coaches or fees and consumption. As each individual cash flow
can be allocated to various stakeholders
17
a wide range of business models could be calculated using the same
tool. Calculations were based on average savings as calculated above also including RUAS and RMS savings as
cost items cannot be divided between these two services (e.g. the same meter is being used). All cash flows are
net present value corrected (NPV) assuming 5% discount rate and each pilot was calculated independently.
Furthermore, no assumptions were made about consumption changes without BECA and estimated resource
price increases were based on past values. To improve comparability, results were modelled for 1,000 dwellings
at each pilot to smoothen effects such as exceptional development cost, economies of scale etc.
The core results for stakeholder and pilot is summarised as the ‘socio-economic’ return. It measures the relative
return for each monetary unit invested. For example, 0% is equal to 1 Euro invested and the total of benefits -
corrected for current (net present) value - is also 1 Euro
18
. Hence, the solution is cost-effective as it pays off all
costs whilst creating benefits (e.g. energy savings). Accordingly, 100% equals a benefit of 2 Euro for 1 Euro
investment (or 20 for 10, 44 for 22 etc.); -50% equals a loss of 50 Cents for each Euro invested etc.
The average additional net cost of deploying the BECA service are 301€ for implementation in the first year and
15€ in operation every year per dwelling. Implementation costs vary depending on already existing infrastructure
whilst operation cost can vary depending on the business model (e.g. contracting models vs. fees).
All pilot sites achieve overall positive socio-economic return within less than 7 years. In some cases the break-
even is almost immediate for the entire pilot site or selected stakeholders. This, however, does not imply the
break-even is achieved for all stakeholders at the same time. Tenants often benefit considerably (>50%, for each
Euro in fees etc., they receive the cost back and at least another 50Cents after ten years). Depending on the legal
framework and business model tenants cover some of the implementation cost either as a lump sum or as yearly
fee in which some of the savings could cover the fee. In some countries no transfer of cost is legally allowed and
then covered by the social housing provider. Social housing providers are also often providing the (new) cost
item of assisting tenants with the information and/or financing the portal delaying pay-off, in some cases
remaining negative in lower absolute figures. IT / measurement provider, usually, share investment cost with
housing providers and/or request fees as part of maintenance / provision contract.
The average total (NPV corrected) benefit created for all stakeholders equals per dwelling and year: Belgrade
33.6€, Darmstadt 156.1€, Havirov 8.53€, Manresa 36.1€, Örebro 219.4€, Ruse 64.3€ and in Turin 73.2€. It has
to be noted that the amount is not always fully allocated to the tenant and that the variance can be partly
explained by differences in income levels (e.g. Sweden and Serbia) and in service provision (combined vs.
RUAS only).
It is advisable to deploy feedback services in environments where smart metering is already existent / obligatory
as the deployment of necessary hardware are the main cost item regardless of whether individualised metering
already existed prior. It is furthermore advisable to combine the provision with RMS at least in the form of
alarms to particularly in sites were wastage goes undetected.
Conclusions
In order to fill the research gap regarding effectiveness and efficiency of ICT feedback instruments we evaluated
the behavioural service of BECA (RUAS) being provided in 1,500 dwellings at seven pilot sites in seven
European countries. Evaluation of effectiveness was carried out on two levels: the level of energy consumption
(heating, hot water and electricity) and the level of energy behaviour related to those consumption types.
The results of multivariate analysis based on pre-post and control group comparisons confirm positive influences
of feedback instruments on energy consumption found in previous studies (e.g. Ehrhardt-Martinez 2010, Fischer
2008, Schleich et al. 2013, Stromback et al. 2011). However, comparability of results across studies is limited
due to differences in interventions, savings calculations and evaluation approaches or targeted consumption
types. BECA tenants, provided with feedback services, saved 0.128 cbm/sqm more hot water, 5.399 kWh/sqm
more electricity and 0.003 Kelvin days/sqm more heat energy than tenants in control groups. Treatment types
also including automated (RMS) services lead to bigger savings with respect to heating showing that the
combination of feedback services and automated features applied on building level is a promising strategy to
17
Tenants, social housing, measurement provider, IT-provider, utilities (not in the role as energy provider but
local infrastructure), city, a freely assigned role and ”other” which is not included in the pilot site results
(subcontractors, workers etc.)
18
Socio-economic return is calculated as: (benefit - cost) / cost = return; whereas benefit and cost are the results
of (at least) the sums across the dimensions: indicator, time and stakeholder.
increase energy efficiency. However, OLS models show rather modest values for adjusted R2 meaning that
further variables not considered in the models (due to small samples sizes) might also influence energy and
resources consumption.
Analysis of energy behaviour provides new knowledge about particular sets of behaviour being influenced by
feedback services and found positive impacts for heating and electricity energy behaviour. Three of four kinds of
behaviour related to heating have been more often improved in the RUAS treatment group than in control group.
The biggest impact was found for turning off the heating when opening the windows being improved by factor
3.315 in the RUAS group compared to the control group. Even higher and for some behaviour statements
statistically significant impacts were found for behaviour related to electricity. Compared to the control group,
the RUAS group improved its behaviour with respect to completely switching off an appliance with Standby-
function by factor 2.7, to unplugging chargers from mains by nearly 5 and to turning off lights when no one is in
the room even by factor 9.7. The treatment group additionally including RMS applied as monitoring basis for
processing RUAS also achieved improvements of behaviour. However, the results suggest smaller impacts
which are probably due to the fact that electricity is also used for heating in some dwellings that received the
combined services. Combined services lead to smaller impacts also for behaviour related to heating. A possible
explanation is some kind of rebound effect: tenants, in buildings equipped with RMS, are less motivated to
change their behaviour assuming RMS will bring about savings regardless of their behaviour. This issue could
be challenged by a more sophisticated communication with tenants better explaining possible achievement of
both service components. Behaviour related to hot water consumption was not visibly improved by the services.
This might be due to the fact that people feel restricted in their comfort when following the saving tips such as
taking a shower instead of a bath.
Furthermore, findings in subgroups suggest that there are user-related – especially motivational factors as well as
local circumstances influencing the success of the services. Tenants without interest in saving energy at home
(hot water) or without energy saving norm before the implementation of the services (hot water, partly
electricity) are interestingly more likely to improve their behaviour. One possible explanation for this is that the
potential for improvement might be higher if tenants did not care and were previously not aware of energy
saving issues before, but the services succeed to raise their motivation. The same assumption can be made for the
higher service impact on behaviour related to heating that has been found for tenants whose heating bills were
not yet calculated based on individual consumption. Possibly due to this, tenants have not been sensitive for their
behaviour, but now realise that there are ways to reduce energy bills. Hence, behavioural feedback services do
not necessarily need to be targeted at persons already interested in the topic. If tenants receive financial support
for rent, results vary between resource types. Related to heating, tenants of the RUAS group without financial
support achieved higher savings, whilst for hot water the opposite is true. This might be dependent from the
pressure the tenants face due to different ways this support is assigned to them or different limits for allocation
of support. Analyses for heating additionally hint at a high impact for tenants of the oldest age group (>59 years)
who seem to be more responsive to the feedback services. A possible explanation is the availability of time to
regularly check the feedback service and to adjust thermostats across the flats.
The Cost-Benefit-Analysis (CBA) has positive socio-economic outcome for all pilot sites and almost all
stakeholders. Depending on the business model applied, tenants either benefit immediately (cost spread over
yearly fees) or after a few years (lump sum). The business model partly depended on whether individualised
metering is already in place. Initial financing by the social housing provider (and contracting models) are the
usual approach to delay payments for tenants. Since the link of consumption and monetary spending is not
necessarily obvious, it is advisable to remind the user about achievements on the portal and in printed bills.
Despite the contribution of the project to investigate energy behaviour in detail which revealed positive
influences of the services for the majority of behaviour, it has to be considered that results are based on rather
small sample sizes leading to results that are not always statistically significant. Furthermore, results of BECA
are restricted to social housing tenants and so might not apply to the wider population. Finally, evaluating the
full impact of behavioural services should also include analyses of the influence of improved behaviour on the
energy consumption which had not been possible within the project due to low sample sizes. In addition to that,
it is still unclear whether behaviour related to hot water consumption generally can hardly be influenced by the
services or whether there are other activities not being considered in BECA such as the duration of taking a
shower or not fully filling the bathtub when taking a bath that can be affected more easily.
Therefore further studies are needed based on even larger sample sizes of a broader target group allowing the
combination of behavioural (survey) data and consumption data. Such studies would also allow drawing
conclusions about different user groups taking into account different social demographic information.
Acknowledgements
This work is based on data collected within the project BECA (www.beca-project.eu), partially financed by the
ICT Policy Support Programme (ICT-PSP) of the European Community. The authors gratefully acknowledge the
contributions of the BECA project partners for implementing the services and supporting such a comprehensive
collection of consumption and survey data.
References
Abrahamse, W. (2007). Energy conservation through behavioral change: Examining the effectiveness of a tailor-
made approach. University of Groningen.
Ajzen, I. (1991). The Theory of Planned Behaviour. Organizational behaviour and human decision processes
50: 179-211.
Darby, S. (2006). The Effectiveness of Feedback on Energy Consumption, Environmental Change Institute,
University of Oxford.
Ehrhardt-Martinez, K., Donnelly A., et al. (2010). Advanced Metering Initiatives and Residential Feedback
Programms: A Meta-Review for Household Electricity Saving Opportunities. Report Number E105.
Washington D.C., American Council for an Energy-Efficient Economy.
European Commission (2012). 2012/148/EU: Commission recommendation on preparations for the roll-out of
smart metering systems. Official Journal L 73, 13.03.2012.
European Union (2012). 2012/27/EU: Directive on Energy Efficiency. Official Journal L 315, 14.11.2012.
Farley, K. and Mazur-Stommen, S. (2014). Saving Energy with Neighborly Behavior: Energy Efficiency for
Multifamily Renters and Homebuyers. ACEEE White Paper. Washington.
Fischer, C. (2008). Feedback on household electricity consumption: a tool for saving energy? Energy Efficiency
1: 79-104.
Froehlich, J., Findlater, L., et al. (2010). The Design of Eco-Feedback Technology. Proceedings of ACM CHI
2010 Conference on Human Factors in Computing Systems: 1999-2008.
Geller, E. S., Berry, T. D., et al. (1990). A conceptual framework for developing and evaluating behaviour
change interventions for injury control. Health Education Research Vol. 5 No. 2: 125-137.
Jackson, T. (2005). Motivating Sustainable Consumption: A Review of the Sustainable Development Research
Network. University of Surrey, Centre for Environmental Strategy, UK.
Pierce, J., Fan, C., et al. (2010). Some considerations on the (in)effectiveness of residential energy feedback
systems. Proceedings of the DIS 2010 Conference: 244-247.
Renz, I., Hacke, U., et al. (2014). Deliverable D7.2 – BECA Pilot Evaluation Outcomes, http://beca-
project.eu/outputs/ (retrieved January 2015).
Rohracher, H. and Ornetzeder, M. (2006). Socio-technical research in the context of ,sustainable technology
programmes'. IFZ - Electronic Working Papers 1-2006.
Schleich, J., Klobasa, M., et al. (2013). Effects of feedback on residential electricity demand – Findings from a
field trial in Austria. Energy Policy 61: 1097-1106.
Schwartz, T., Stevens, G., et al. (2014). What People Do with Consumption Feedback: A Long-Term Living Lab
Study of a Home Energy Management System. Journal of Interacting with Computers (IWC): 1-26.
Steg, L. and Vlek, c. (2009). Encouraging pro-environmental behaviour: An integrative review and research
agenda. Journal of Environmental Psychology 29: 309-317.
Stromback, J., Dromaque, C., et al. (2011). The potential of smart meter enabled programs to increase energy
and systems efficiency: a mass pilot comparison (Short name: Empower Demand), VaasaETT, Global
Energy Think Tank.
Vogt, G., Dashja, E., et al. (2012): Deliverable D5.2 – BECA Pilot Operation, http://beca-project.eu/outputs/
(retrieved January 2015.
Vogt, G., Dashja, E., et al. (2013): Deliverable D8.3 – BECA Interim Report on Dissemination and Exploitation
Activities, http://beca-project.eu/outputs/ (retrieved January 2015)
Weiss, M., Staake, T., et al. (2012). PowerPedia: changing energy usage with the help of a community-based
smartphone application. Personal and Ubiquitous Computing 16: 655-664.
