![Design-Interference model for pervasive IS [adapted from Brandt et al. 2017]](https://www.researchgate.net/profile/Matthias-Hauser-4/publication/319998755/figure/fig4/AS:631652556476432@1527609113175/Design-Interference-model-for-pervasive-IS-adapted-from-Brandt-et-al-2017_Q320.jpg)
Content uploaded by Matthias Hauser
Author content
All content in this area was uploaded by Matthias Hauser on Dec 13, 2017
Content may be subject to copyright.
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 1
Designing Pervasive Information Systems:
A Fashion Retail Case Study
Completed Research Paper
Matthias Hauser
University of Würzburg
Josef-Stangl-Platz 2
97070 Würzburg, Germany
matthias.hauser@uni-wuerzburg.de
Sebastian A. Günther
University of Bamberg
Kapuzinerstraße 16
96047 Bamberg, Germany
sebastian.guenther@uni-bamberg.de
Christoph M. Flath
University of Würzburg
Josef-Stangl-Platz 2
97070 Würzburg, Germany
christoph.flath@uni-wuerzburg.de
Frédéric Thiesse
University of Würzburg
Josef-Stangl-Platz 2
97070 Würzburg, Germany
frederic.thiesse@uni-wuerzburg.de
Abstract
We expand the design theory for cyber physical systems by introducing the notion of
mutability of legacy components. Mutability refers to the extent components of the legacy
system can be modified to facilitate better integration with the information system
components. In particular, immutable components must remain untouched during the
introduction of a cyber physical system. We explore our propositions considering a design
showcase for an automated checkout system for retail fashion environments. The artifact
consists of an RFID sensor infrastructure and data-driven software components that
process the low-level sensor data to provide seamless checkout functionality. The system
is evaluated by means of a comprehensive trial in a representative retail laboratory.
Keywords: Radio Frequency Identification (RFID), Design Science, Cyberphysical
Systems, Data Analysis, Automated Checkout, Internet of Things
Introduction
The term "cyberphysical systems" refers to a tight integration of and coordination between computational
and physical resources. Thereby, these systems can greatly enhance efficiency, functionality and reliability
of previously non-digitized systems (National Science Foundation 2010). This creates transformative
opportunities for information systems across various economic sectors (Borgia 2014). In manufacturing,
industrial internet applications turn shopfloors into smart factories. In the automotive sector, ride-hailing
platforms (Uber, Lyft) as well as innovative manufacturers (Tesla, Waymo) are giving incumbent OEMs a
run for their money by replacing traditional cars by fleets of shared, autonomous vehicles (The Economist
2016). Smart grids are reversing the legacy supply-follows-demand paradigm of power systems to enable
greener and more reliable electricity supply (Amin and Wollenberg 2005; Blumsack and Fernandez 2012;
Farhangi 2010). Healthcare innovations (wearables, augmented surgical tools) will increase well-being and
health outcomes of generations to come (Lee and Sokolsky 2010). New retail solutions (automated
checkout, smart store environments) help create a better shopping experience for customers
(Kourouthanassis and Roussos 2003).
Cyberphysical systems have progressed beyond visions and technical pilots and instead are increasingly
becoming part of everyday life. In line, research activities are no longer exclusive to the electronics and
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 2
computer science field. Rather, they have migrated into applied fields such as information systems research.
To better understand the potentials of these systems and support their application, a more comprehensive
understanding of their special design requirements is needed. Brandt et al. (2017) highlight the challenge
of enhancing “technical components […] inherent to the legacy system” with information systems. In
particular, they put forward enhancing features (how IS upgrades the capabilities of the legacy system) and
interferences (obstacles and challenges faced when designing the IS) as key building blocks of IS design for
cyberpyhsical systems.
We want to expand this design theory by highlighting the multi-facetted nature of physical components. In
particular, we introduce the notion of mutability of legacy components. Thereby we highlight that some
legacy components can be modified during digitization (mutable components), while others must remain
untouched (immutable components). To this end, we adapt the Brandt et al. (2017) design approach and
substantiate our propositions, considering a pervasive information system in a retail environment. Our
showcase is an automated checkout system for fashion retail. This is a particularly suitable showcase as it
features an environment with (i) limited process control (e.g., unpredictable customer behavior) and
(ii) established processes (i.e., established customer behavior patterns).
Component Mutability in Pervasive Systems
Cyberphysical systems emerge as upgrades of previously existent techno-social system. Consequently, they
need to be “woven” around the legacy system (Weiser 1991). We refer to components of the legacy system
as mutable if they can be modified to facilitate better integration with the information system components.
On the other hand, immutable components will function in a cyberphysical system the very same way as in
the original system. Importantly, immutability can arise from both physical (infrastructure) as well as non-
physical components (business processes, customer behavior) of the legacy system. Non-physical
components are of particular interest for systems interacting with humans (e.g., smart retail environments,
self-driving cars confronted by human-driven cars, human-robot interaction in factories) where key
components remain outside of the realm of the system designer. This relates to the special case of pervasive
systems which are “digital environments that are sensitive, adaptive, and responsive to human needs” (Saha
and Mukherjee 2003).
Immutable Physical Components
The physical environment (e.g., factory shop floor, a retail store in a mall or road infrastructure) in which a
pervasive information system is implemented is often immutable. A case in point are architectural
constraints, which would pose prohibitive costs for modifications. Similarly, lack of space can rule out
certain system configurations. Consequently, system designers will seldom be able to conceive an “ideal”
system from scratch but rather have to identify the most promising upgrade path.
Immutable Non-Physical Components
Many legacy environments feature well-established processes (e.g., sales floor environments or
transportation systems). The introduction of pervasive systems in such environments allows only slight
changes of these processes to ensure stakeholder acceptance. To this end, pervasive systems “should be [...]
objectively visible but subjectively invisible” (Chalmers et al. 2003) in these settings. Accounting for the
immutability of components has direct ramifications for the design objectives and the evaluation criteria of
pervasive systems. Figure 1 sketches how accounting for the different component types transforms the
design approach put forward by Brandt et al. (2017).
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 3
Figure 1. Design-Interference model for pervasive IS [adapted from Brandt et al. 2017]
Pervasive Retail Systems
In the following we focus on pervasive retail systems (Kourouthanassis et al., (2003). These are
characterized by a high prevalence of immutable components, both physical (stores have limited space and
a legacy layout) and non-physical components (customers have a clear expectation of how a retail store
functions and are unlikely to internalize drastic changes). Consequently, this is a particularly suited setting
to substantiate the conceptualized relationships. In particular, we focus on checkout systems. Traditional
checkout systems are labor-intensive and can be a great source of frustration for customers when having to
wait in line (Manyika et al. 2015).
Self-service checkout systems
To reduce costs, retailers have started adopting self-service technologies which enable shoppers to scan,
bag and pay for their purchases with little or no help from store personnel (Litfin and Wolfram 2006; Orel
and Kara 2014). These systems, however, offer hardly any improvements over traditional checkout with
respect to the customer experience, potentially creating new challenges as many customers consider the
service as frustrating, irritating and alienating (Meuter et al. 2000).
1
Self-service checkout systems can be
roughly categorized into (i) stationary systems at store exits and (ii) portable systems (e.g., handhelds,
mobile phones) that customers carry with them during the shopping tour through the store. Both types of
systems usually rely on linear or matrix barcodes (e.g., QR codes). The first group comprises self-checkout
terminals (e.g., NCR self-checkout systems) and tunnel scanners (e.g., Wincor Nixdorf 360-degree
scanners). In the first case, customers have to scan the items that they want to purchase one after the other
by themselves. Tunnel systems, on the other hand, rely on camera systems that scan the barcodes of items
on a conveyer belt and customers thus simply have to put their items on the belt. In contrast to stationary
systems, portable systems allow continuous scanning of items while customers are walking through the
store. Such portable systems can be handhelds that retailers provide to their customers as well as the
1
Meuter et al. (2000) found that causes of dissatisfaction with self-service technologies were failure of the
technology, design problems in regard to both the technological interface and the service that it offered, as
well as customer-driven failures, e.g., forgetting one’s personal identification number.
Pervasive Information System
Sensor
infrastructure
Software
components
Mutable
Immutable
IT artifact
Design features
Enable
Intended
affordance
Enhance
Legacy components
Design
outcome
Design
objectives
Adjust
Accommodate
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 4
customers own mobile phone whereat the latter case requires that customers install an app that provides
self-checkout functionalities.
Automated Checkout Systems
In contrast, fully automated checkout systems scan, total and charge a customer’s purchases to a registered
payments account while customers are simply leaving the store. These systems promise more sales because
of increased customer experience and costs savings because less store personnel is needed.
Such a system must tackle two subtasks: The systems (i) have to reliably detect purchased products and
(ii) assign these to individual customers. Undetected products cause direct losses to the retailer and
inventory inaccuracy resulting in disruptions in the replenishment process and severe out-of-stock
situations (Kang and Gershwin 2005). Incorrectly assigning items to a customer, on the other hand, may
lead to customer dissatisfaction and interruptions of in-store operations (Hayes and Blackwood 2006).
The literature on fully automated systems is sparse. To the best of our knowledge, details of only two
systems that address the previously mentioned challenges have been circulated. The first system
(“MyGrocer”) relies on shopping carts equipped with RFID readers that detect objects placed in the carts
(Kourouthanassis and Roussos 2003; Roussos et al. 2003). As customers have their own RFID-equipped
shopping carts during a shopping tour, the assignment of products to customers is a somewhat trivial task;
customers get charged for the products that the RFID reader of their shopping cart detected. The second
system is “Amazon Go,” which received enormous attention in the media over the last months. The system
promises to automatically (i) detect products taken from or returned to shelves, (ii) keep track of the
products chosen by customers in virtual shopping carts, and (iii) charge the customers’ Amazon accounts
after they leave the store. In addition, Amazon promises that all customers need to use their system is an
Amazon account, a supported smartphone, and the Amazon Go app to register when entering the store
(Amazon 2016). Available information suggests that the system stores the inventory locations of all
products within Amazon Go stores and mainly relies on cameras to detect products that customers take
from or return to particular inventory locations.
2
In addition to the cameras, additional sensors (e.g.,
pressure sensors, infrared sensors, light curtains, and RFID readers) as well as customer information (e.g.,
purchase history) can be utilized to identify and assign purchases.
Checkout System Taxonomy
We propose the framework presented in Figure 2 to distinguish the different automated checkout systems
that we just presented. We first differentiate between manual and automated purchase initialization.
Second, we differentiate between systems that require scanning of products at the store exit (i.e., stationary
systems) and systems that require scanning at the very moment customers pick them from shelves or put
them into shopping carts. Figure 2 reveals that we did not identify fully automated stationary checkout
solutions. Self-checkout terminals shift the checkout process from the employee to the customers and we
do thus not consider them as fully automated systems. Even though the degree of automation of tunnel
systems is higher in comparison to self-checkout terminals, these systems do not allow customers to leave
the store without pausing because they still have to put all the items that they want to purchase on a
conveyer belt and wait until they got scanned by the tunnel system.
2
Although Amazon has not published any technical details about their system, information on the
company’s website and two patents filed by the company (Kumar et al. 2015; Puerini et al. 2015) provide
insights into the implementation of this pervasive retail system.
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 5
Figure 2. Taxonomy of automated checkout systems
Automated Checkout Design for Fashion Retail Stores
Brick-and-mortar fashion stores are facing increasing competition from online retailers. Pervasive retail
systems can increase the attractiveness of these stores and can help to reduce disadvantageous labor
overhead of stationary stores. Thereby, they can increase their competitiveness in the battle between high
street and online retail. To this end, we collaborated with a major fashion retail chain in order to develop
and test an automated checkout prototype for fashion stores. Our objective is the design of a system that
can be integrated into existing stores without expensive modifications of the store infrastructure and does
not require changes of established customer processes within fashion retail stores.
Requirements Analysis
The above-described automated checkout solutions were developed for supermarket settings. Fashion
stores significantly differ from grocers with respect to in-store processes and technology availability. We
put forward the following observations and explain how they affected the design decision:
1. Fashion retail stores do not feature shopping carts or baskets.
We consider this an immutable process of fashion retailing, as customers will likely be alienated by a
store requiring them to use shopping carts to track their purchases. Furthermore, store layouts may not
permit carts to navigate the shopping area (immutable physical component). Lastly, the mental
association of bulk shopping arising from carts and baskets may be detrimental to brand image
(immutable non-physical component). This rules out an automated checkout system based on smart
shopping carts.
2. Customers in fashion retail stores usually leave unwanted garments at the changing room.
Again, this business process is considered immutable as customers will not accept running back to
search for the shelf where they removed a garment. This rules out an automated checkout system which
relies on shelf activity to track purchases.
3. Usage of cameras is problematic in key areas of fashion stores (i.e., changing rooms).
Customer privacy is immutable and thus rules out an automated checkout system, which relies on video
surveillance to track purchases.
Scanning at store exit
(Stationary systems)
Continuous scanning
Automated
purchase
initialization
Manual purchase
initialization
- Self-checkout terminal
- Tunnel scanner Self-scanning (with handheld
or mobile phone)
- Smart shopping carts
- Smart shelves
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 6
4. Major fashion retailers have implemented item-level RFID tagging of products.
3
Leveraging the sensor infrastructure offered by the legacy system facilitates a cost-effective and less
intrusive integration of the pervasive checkout system.
Consequently, we decided against smart shopping carts or smart shelves. Instead, we focused on the lower
left corner of Figure 2: An automated system with a central point of scanning (i.e., items are detected when
customers leave the store). Furthermore, we opted for RFID-based item detection. However, RFID-based
identification is more challenging than in the MyGrocer project where carts only need to detect items within
them. In our case, the system needs to detect items that leave the store through an exit gate. This requires
antennas with large read range and high power. Unfortunately, this in turn leads to the detection of RFID
tags carried nearby by the gate instead of through the gate.
4
Furthermore, assigning items to individual
customers is impossible with a purely hardware-based approach unless customers wait in line and pass the
gate one after the other to avoid purchases of different customers being read at the same time.
In the following, we highlight how we mitigated these challenges by means of an information system based
on predictive algorithms processing online sensor data streams.
System Architecture and Infrastructure
The architecture of our automated checkout artifact combines hardware and software (see Figure 3). The
hardware consists of two RFID reader installations, a ceiling-mounted system that tracks items in the store,
and a gate-mounted system that helps to detect items that are leaving the store. This infrastructure collects
low-level RFID data that is then processed by the software components. There are two distinct software
functionalities. The first software component uses data mining techniques to reliably and timely detect
items that are leaving the store. The second software component assigns items leaving the store (identified
by the first component) to individual customers. To this end, we infer item paths in the shopping area and
then apply cluster analysis to group them. The procedure rests on the assumption that the paths of items
purchased by one customer are more similar to each other than to paths of other items. The output of the
artifact are the individual shopping baskets of customers.
Figure 3. Architecture of the Automated Checkout Artifact
Figure 4 depicts the infrastructure with the two parallel RFID readers from the company Impinj, a Seattle-
based manufacturer of RFID devices and software. The gate-mounted system features four far-field
antennas (Impinj Inc. 2017a), the ceiling-mounted system an array with 52 far-field antenna beams
mounted in one housing (Impinj Inc. 2017b).
3
RFID identifies products at the item level without a direct line of sight. Furthermore, it facilitates
simultaneous bulk detection of multiple objects.
4
In logistic processes (e.g., in distribution centers) employees are often instructed not to approach gates
without the intention of walking through them. However, such instructions can hardly be imposed on
customers while blocking generous areas around the gate would reduce the usable store area.
Product
detection Product
assignment
Processing
Customer
shopping
baskets
Input
Output
RFID
infrastructure
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 7
Figure 4. Infrastructure with two parallel RFID reader installations
Table 1 provides an exemplary data excerpt from the raw data gathered with the RFID infrastructure. Each
row reflects a single tag read event triggered by one of the readers’ antennas. Here, EPC is the unique
identifier of the RFID tag, RSSI indicates the radio signal's power, phase angle is the current state of the
backscattered sinusoidal wave, and Antenna is the ID of the antenna that read the tag.
Table 1. Exemplary data excerpt
EPC
Timestamp
RSSI
Frequency
Phase
Power
Reader
Antenna
E28011606…0001
1462975965.76
-70.0
866.9
6.25
30
Ceiling
22
E28011606…0001
1462975965.77
-70.5
866.9
0.04
30
Ceiling
22
E28011606…0002
1462975965.79
-69.0
865.7
2.72
31.5
Gate
4
E28011606…0003
1462975965.80
-67.0
866.9
0.81
30
Ceiling
49
E28011606…0003
1462975965.81
-67.0
866.9
0.87
30
Ceiling
49
Item Detection Approach
The software component for product detections should facilitate reliable and timely identification of RFID-
tagged objects. To this end, we seek to develop a data-mining model that distinguishes in real-time between
products that are moved through the RFID gate and others (see Figure 4). In addition, we aim at detecting
products at the very moment they are moved through the gate (i.e., when a person leaving the store is stand
right in the middle of the RFID gate). The underlying approach builds upon previous research presented at
the International Conference on Information Systems in 2015 (Hauser et al. 2015). In this article, we also
describe a classification artifact able to distinguish between RFID-tagged objects that are carried through
an RFID gate and others. In the present paper, however, we go beyond previous research by considering
run fragments in real-time instead of evaluating completed paths ex-post.
Figure 5 shows the conceptual approach we follow in the development of the data-mining model. The
approach loosely follows the Cross-Industry Standard Process for Data Mining (CRISP-DM), a standard
process model widely applied in data-mining projects (Chapman et al. 2000).
RFID gate with reader and gate-
mounted antennas
Shopping area
Ceiling-mounted RFID reader
with 52 far-field antenna beams
Store exit
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 8
Figure 5. Development of data mining model
To enable continuous evaluations in real-time, we first apply a sliding window approach. A sliding window
is a window of certain size (e.g., detections of the last two seconds) that gets updated at certain, regular time
intervals (Jeffery et al. 2006). We propose (i) window sizes of two seconds to capture enough information
to solve the classification task reliably and (ii) window shifts every 250 milliseconds to facilitate real-time
evaluation. It should be noted, that every window contains only detections from one particular tagged
product within reading range of the antennas.
In a second step, we examine the two-second windows and extract so-called “features” from the raw data,
which contain information regarding observed real-world events. The considered features are appropriately
specific to RFID and must be developed based on knowledge of the particular business process.
We engineered 184 different features for training of the classification models. One example of a feature with
high predictive power is the mean RSSI value measured in a series of detections of a particular tag within
the two-second windows. Here, we first consider the reader level and derive a mean RSSI value for the gate
antenna detections and one for the ceiling antenna detections. In addition, we zoom in on the individual
antenna level and derive values for detections of the antennas. Another example are the parameters of a
Gaussian fit of the signal strength values for detections of a particular tag within the two-second windows.
We approach the classification problem considering logistic regression, as well as the black-box models
artificial neural networks (ANN) (Bishop 2006), support vector machines (SVM) (Chang and Lin 2011), and
gradient tree boosting (XGBoost) (Chen and Guestrin 2016). In each case, we tested different parameter
settings (e.g., numbers of hidden layers and nodes or maximum number of constructed trees) to ensure
good and robust classification results.
Every 250 milliseconds, the data-mining model considers two-second windows of raw data for every tagged
product within reading range of the antennas and analyzes whether the particular tags are currently moved
through the gate or not. To detect a product that is moved through the gate, the model has to classify at
least one of the associated two-second windows as moving through the gate (true positive event). In this
context, associated windows are all the windows that contain detections from a particular product that were
built while the product was moved out of the store (i.e., while the product was within reading range of the
antennas). In contrast, to avoid false alarms (false positive events), the model has to classify none of the
two-second windows associated with detections from products that are in vicinity of the gate but are not
moved through it (e.g., products that are carried by the gate or products on shelves close to the gate) as
moving through the gate.
Purchase Assignment Approach
The software component for product assignments assigns items leaving the store (identified by the first
component) to individual customers. To this end, we first infer item paths in the shopping area and then
apply cluster analysis to group them. The procedure rests on the assumption that the paths of items
purchased by one customer are more similar to each other than to paths of other items.
Item Paths Determination
We rely on state-of-the-art indoor localization techniques to infer item paths. To this end, we apply the
“Scene Analysis” technique to estimate the position of an object by matching its real-time measurements
with the raw data “fingerprints” at different positions (Liu et al. 2007). For the development of the data-
mining model, we again follow the conceptual approach outlined in Figure 5. We again consider a sliding
window approach with window shifts every 250 milliseconds to facilitate continuous evaluation. In contrast
to the development of the first software component, we do, however, not rely on window sizes of equal
length but split the data such that each chunk contains only detections from one collection cycle covering
all 52 successively activated antenna beams of the ceiling-mounted RFID reader. The durations of the
Split raw data into
data chunks
Develop features
with high
predictive power
Train
classification
models
Evaluate
classification
models
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 9
physical cycles depend on the number of tags in the antenna field and therefore vary over time. Considering
time intervals of equal length would have the drawback that some antenna beams might not have been
activated yet. This, in turn, would lead to areas not covered by the system resulting in undetected items. In
the artifact’s first software component we consider time intervals instead of collection cycles because objects
that are carried out of the store are definitely detected by the gate antennas (in contrast to objects that are
somewhere within the shopping area). Whereas for the localization of RFID-tagged objects the data from
the ceiling antennas is decisive, for the identification of objects that pass the gate the gate antennas are
more important.
We developed 174 features for the training of the classifiers that help in localizing tags within reading range
of the antennas. Most of them are antenna-based features from the ceiling-mounted RFID reader but we
also leverage the low-level data from the gate antennas. For instance, a high maximum signal strength from
the gate antennas in combination with a low number of reads from the ceiling-mounted reader is a good
indicator that an object is very close to the exit. Intuitively, the high maximum signal strength indicates that
the person is near the gate, the low number of reads that the person is faced backwards to the ceiling -
mounted system (i.e., that the person’s body is shielding the RSSI signals).
We apply multiclass classification for solving the localization task, which necessitates dividing the shopping
floor area in front of the gate into grid fields and collecting training data for each of these fields (raw data
“fingerprints”). Here, the number of grid fields denotes the number of classes considered in the data-mining
model. We consider the same machine learning models as for the first software component and again tested
different parameter settings to ensure good and robust classification results. To determine item paths, we
concatenate the most probable locations of individual items over time.
Assignment Process
To assign RFID tagged items to customers, the automated checkout artifact needs to identify the correct
customer for the items that are currently leaving a store. Thus, the task is to group the items within the
antennas’ reading field (i.e., the shopping floor area in front of the RFID-equipped gate) such that items in
the same group belong to the same customer. We approach the problem by first determining all individual
item paths within the antennas’ reading range. The procedure for the assignment of items then rests on the
assumption that paths of items carried by one customer are more similar to each other than to paths of
other items.
Figure 6 illustrates the actual assignment process. The process is triggered every time the first software
component detects a product that is moved through the gate. The objective then is to determine the items
that belong to the shopping basket of the particular item. To this end, we analyze the paths of all items in
the antennas’ reading field. We first determine whether all the items belong to a single customer by applying
a simple threshold rule based on the average Euclidean distance between pairs of items. If all items belong
to one customer, we assign them to one customer shopping basket. Otherwise, we use clustering techniques
to determine the items that belong to the item that led to the “through the gate” event. If the first software
component triggers another “through the gate” event, we repeat the process. This time, however, we exclude
items that are already assigned to customer shopping baskets.
Figure 6. Visualization of the process for the assignment of objects to customers
More than
one
customer?
Perform
clustering
“Through
the gate”
events?
Exclude
assigned
objects
Yes
Yes
No
No
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 10
We follow a two-step approach to group items. We first determine clusters for every possible number of
customer shopping baskets and evaluate each clustering result. Then, in a second step, we choose the best
result. To determine the item groups, we use the Partitioning Around Medoids (PAM) clustering algorithm
(Reynolds et al. 2006). For the evaluation of the similarity between pairs of tagged items, we again rely on
the Euclidean distance. For the evaluation of the goodness of the clustering results, we calculate for each of
them the average silhouette width, which indicates whether objects are matched well to their own clusters
and poorly to neighboring clusters (Rousseeuw 1987).
Evaluation Setting
We collected large data sets in the laboratory under real-world conditions for the evaluation of the
automated checkout artifact. To this end, we set up an experimental shopping area in a retail research
laboratory. The dimensions of our experimental shopping area are 4.8m × 4.8m. For the collection of
training data for the indoor localization model, we divided this area into 64 grid fields of equal size.
The artifact design necessitates on the one hand the collection of RFID raw data traces stemming from tests
with people that carry RFID-tagged objects and simulate real world customer movements in the
experimental shopping area. In addition, we need raw data fingerprints at different location within the
shopping area for training and testing of the indoor localization data-mining model. We collected raw data
fingerprints for all 64 grid fields within the experimental shopping area. To achieve this, a person carrying
garments stood in the shopping area and held the garments such that they were positioned right above one
of the fields. During the tests, the garments were moved up and down to reflect real-life shopping situations.
We collected approximately two minutes of low-level RFID data for every grid field and two different
numbers of tagged items (one and three objects). The resulting RFID data set comprises 1,515,918
individual tag reads. Whereas the first data set is used for model training and evaluation of the artifact, the
second data set is only used for model training (i.e., training of the indoor localization model).
Our experimental setup takes into account the limited process control at store exits by considering multiple
walking paths, different numbers of persons and RFID-tagged items as well as different movement speeds
(i.e., walking and running). Figure 7 illustrates the customer movement paths that we consider in our
analysis.
Figure 7. Test setting with typical customer movement paths
Error sources that we identified during our experiments are (i) customers with tagged objects that walk by
the gate in close proximity and (ii) customers with tagged objects that leave the store at the same time on
similar movement paths. Therefore, we expanded our analysis to include such settings. In total, our
experimental design includes 18 tests and we repeated each of them 50 times. Table 2 provides a complete
overview of the experimental design. The final data set comprises 1,500 runs with a total of 1,431,347
individual tag read events.
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 11
Table 2. Experimental Design
Persons
Tags
Speed
Movement patterns
A
B
C
D
E
F
G
H
I
J
K
L
1
3
Walking
50
50
50
1
3
Running
50
50
50
1
6
Walking
50
50
50
2
6
Walking
50
50
50
50
50
50
3
9
Walking
50
50
50
As training and testing of data mining models necessitates labelled data, we additionally installed a light
barrier at the gate for the data collection process to identify the exact time a tag was moved through the
gate. We did not use the information from the light barrier for the development of our features and it is thus
not necessary for real-world implementation of the artifact. We decided against using the information
because (i) our objective is the development of an artifact that facilitates automated checkout with as little
hardware investment as possible and (ii) the light barrier requires direct light of sight and is thus very
susceptible of faults in real-world implementations.
The artifacts’ software components rely on supervised machine learning techniques. We thus have to train
the models to instantiate the artifact. We use data stemming from the tests with typical movement paths in
retail stores for the training of the product detection component’s underlying data mining model. Therefore,
we first split the low-level data streams into data chunks (i.e., windows) and calculate the features for each
of them (see Section Item Detection Approach). In addition, we use the fingerprints for every grid field
within the experimental setting for the training of the indoor localization model. To this end, we first split
the low-level data streams into collection cycles and calculated the features for each of them (see Section
Purchase Assignment Approach). We then train a classification model with one class for each of the 64 grid
cells. We use the collected fingerprints only for the training of the indoor localization model and not for the
evaluation of automated checkout artifact.
Evaluation Results
The evaluation of the automated checkout artifact is based on the tests with typical movement paths in retail
stores. We perform 5-fold cross validation to ensure representative results: In each round, we use 80% of
the data for the instantiation of the product detection component’s underlying data mining model and use
the remaining 20% for the evaluation of the automated checkout artifact. We first evaluate the system’s
ability to (i) reliably and (ii) timely detect items that are moved through the RFID gate. Subsequently, we
evaluate the assignment of purchases to customers.
Reliability of Detection
In our tests, 4,350 items (1,300 customer shopping baskets) were carried through the gate and another 600
items (200 customer shopping baskets) were carried around but did not leave the shopping floor area (see
movement patterns I, K and L in Figure 7). We base our evaluation of the model reliability on the criteria
Balanced Accuracy, Precision and Recall. As described in Section Item Detection Approach, items are
classified as “moved through the gate” if the model classifies at least one of the associated two-second
windows as moving through the gate. The performance indicators for the four types of classifiers are
summarized in Table 3. Balanced accuracy is the arithmetic mean of the detection rates of both classes,
Precision the share of instances classified as “moved through the gate” that actually were moved through
the gate. In our application, if we erroneously classify tags that were not moved through the gate as "moved
through the gate" (false alarms), precision is diminished. Recall measures the proportion of correctly
classified “through the gate” instances. For very conservative classifiers that tend to classify instances as
“not through the gate” in uncertain cases, recall will be low. With the exception of the logistic regression
model (LogReg), all models achieve a high level of classification performance. Recall values of 99.93%
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 12
(SVM), 99.79% (XGBoost), and 99.98% (ANN) indicate that the model classified almost all items that were
moved through the gate accordingly. A detailed analysis of the false positive classifications (false alarms)
revealed that most errors were caused by falsely classifying items that were carried by the gate with very
close proximity (see movement pattern K in Figure 7).
Recall values below 100% on item level (see Table 3) do not necessarily imply that some items might not
get assigned to customers’ shopping baskets. This is because the item detection component only needs to
classify at least one of the items in a shopping basket as “through the gate” to trigger the assignment process
for the items that are currently within reading range of the antennas. To obtain a more accurate evaluation
of the item detection component, we therefore additionally consider classification results on basket level.
Table 4 presents the evaluation results. Baskets are correctly classified as “moved through the gate” if at
least one item in that basket was correctly classified as “moved through the gate”. Accordingly , the
component correctly identifies shopping baskets that did not leave the shopping floor if it never classifies
any of the items in those baskets as “moved through the gate”. With greater than 94% balanced accuracy,
greater than 98% precision, and 100% recall the SVM achieves the best classification results. The 100%
recall indicate that the model detected all the shopping baskets that were moved through the gate.
Table 3. Item-level classification results
Classifier
Balanced
Accuracy
Precision
Recall
SVM
97.05%
99.20%
99.93%
XGBoost
97.31%
99.29%
99.79%
ANN
92.41%
97.95%
99.98%
LogReg
85.11%
98.43%
79.38%
Table 4. Basket-level classification results
Classifier
Balanced
Accuracy
Precision
Recall
SVM
94.00%
98.19%
100.00%
XGBoost
93.50%
98.04%
100.00%
ANN
87.50%
96.30%
100.00%
LogReg
86.58%
96.23%
98.15%
Timeliness of Detection
Apart from reliability, the timeliness of shopping basket detections is important. If the shopping basket of
a customer is detected after the customer walked through the RFID gate, it might be too late to initiate a
payment process. Payment process initiation long before the customer is actually walking through the gate,
on the other hand, could also be a potential error sources because these customers might not have made up
their mind yet and on their way to the exit decide not to leave the store. Figure 8 visualizes the temporal
distribution of the detections of the individual shopping baskets.
The detection time is the time difference between the time the item detection component correctly classified
the shopping basket as “moving through the gate” and the time the light barrier was triggered by the
customer carrying the particular basket. The histograms and the boxplots reveal that the classifiers indeed
detect most baskets before customers walk through the gate. With the exception of the logistic regression
model, all the 97.5% percentile values of the classifier’s detection times are negative, which indicates that
at least 97.5% of the baskets were detected before the light barrier was triggered. With the earliest detection
at -1.68s, a 2.5% percentile value of -0.73s, a median detection time of -0.39, a 97.5% percentile value of
-0.12s, and the latest detection at 0.05s, the SVM arguably achieves the best results. As the SVM, as
described above, also showed to be the most reliable classifier for solving the classification task, we choose
the classifier for our automated checkout artifact.
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 13
Figure 8. Detection time histograms and boxplots with 2.5 and 97.5 percentiles
Purchase Assignment
Every time a basket is detected, the purchase assignment component determines the items that are in the
basket considering the paths of all items within the shopping area. Table 5 presents the evaluation results
for the different movement patterns in our experiment and the different classifiers that we considered for
indoor localization of RFID tagged items. The results indicate that the component assigns most purchases
to customers correctly if we use XGBoost or ANN for indoor localization. In both cases, the
misclassifications arise in two particularly challenging test scenarios where multiple customers approach
the exit gate simultaneously on very similar (almost identical) movement paths. In the first case (movement
pattern I) two customers approach the gate next to each other but one of them is turning to the right just
before the gate and walking by the gate. Here in some of the tests, the component assigns items of the
customer not leaving the store to the customer leaving the store. In the second case (movement pattern J),
three customers with very similar movement paths leave the store next to each other at the same time which
results in some items being assigned to the wrong customers.
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 14
Table 5. Correctly assigned purchases
Classifier
A-C
D
E
F
G
H
I
J
K
L
XGBoost
100%
100%
100%
100%
100%
100%
54%
62%
100%
100%
ANN
100%
100%
100%
100%
100%
100%
48%
20%
100%
100%
SVM
100%
100%
100%
100%
82%
100%
14%
26%
100%
98%
LogReg
100%
96%
98%
78%
92%
96%
12%
14%
90%
70%
Expected Contribution and Future Work
The present study expands theory on the design of cyberphysical systems by introducing the notion of
mutability of legacy components. Mutable components are components that can be modified during the
digitization of physical objects. On the other hand, immutable components must be integrated the way they
currently are. Immutability arises both from physical (architectural constraints) and non-physical (business
processes) components of the legacy system. We illustrate this idea by considering an automated checkout
system for retail fashion environments as showcase. This is a particularly suitable showcase for our study
because it features an environment with immutable physical components (e.g., store layout) and immutable
non-physical components (i.e., established customer behavior patterns). Finally, with our automated
checkout artifact we demonstrate how data mining models can be leveraged to assign RFID-tagged products
to customers. We apply such models to (i) detect items that are currently leaving a fashion store through an
automated checkout gate and (ii) assign these items to the correct customers. The data mining models allow
us to cope with the immutable physical conditions on the retail sales floor (e.g., unpredictable customer
movement behavior). Although most shopping baskets assigned by the artifact contained the correct items,
we found misclassifications in two particularly challenging test scenarios. In these scenarios, multiple
customers approach the exit gate simultaneously on very similar (almost identical) movement paths. In
practice, such a situation could easily arise when friends are shopping together which highlights the
limitations of the pilot implementation. Nonetheless, our study demonstrates the fundamental feasibility
of RFID-based automated checkout.
We see various potential for model improvements that might enable us to distinguish between customers
even if their movement paths are very similar. First, we propose to use probabilistic models to improve the
accuracy of item paths. Instead of only concatenating most probable locations of individual items over time,
we suggest to consider the layouts of store areas and characteristics of processes within retail stores to
improve path accuracies. The former comprise for example information about the location of shelves, the
latter builds on the assumption that some sequences of item locations within a certain time are more likely.
In Hauser et al. (2017) we already demonstrated the potential of leveraging such information considering a
similar fashion retail store application. Second, we see potential for leveraging additional data sources for
improvement of the assignment process. One possibility is the implementation of additional sensor systems
(e.g., camera systems). In case, it might not be technologically feasible to reliably assign items to customers
(even with additional hardware), we propose to include data from additional information systems into the
assignment process (e.g., customer purchase history, sales data, and garment characteristics). This
approach is in line with Lee (2008), who suggests that in such cases “the next level of abstraction […] must
compensate with robustness.” Product characteristics, for example, could be very helpful in case two
customers of different height or opposite sex leave the store at the same time (as these are very likely to
carry very different products with them). Finally, a very promising approach is extending the store area
within reading range of the system by installing additional ceiling mounted RFID systems in the store. The
more physical area can be observed by the system the more likely different item movement paths can be
distinguished.
Going forward, we want to enhance the (i) generalizability of the proposed fully automated checkout artifact
and (ii) extend our system to form an entire system, i.e. a pervasive retail store, instead of considering
individual system components. The first objective comprises evaluating the fully automated checkout
artifact in different environments (i.e., different laboratory, showroom at one of our industry partners, and
ultimately real-world implementation) and expand the test setting to scenarios that are more complex (e.g.,
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 15
scenarios in which customers take objects from shelves that are placed near the exits). We expect that such
tests will introduce new challenges that require improvements of the developed data mining models.
Pervasive systems will not be implemented to augment a singular business process. Instead, they should
fuel a transformation of the entire system that allow enriching physical environments with online services.
Information about item paths, for example, cannot only be leveraged to distinguish customers at the
checkout gate. Instead, the information could, for example, also be used to provide customers in fitting
rooms with product recommendations that consider their journeys through the store (e.g., did the customer
spend a lot of time in a particular section of the fashion store? Which items are often tried on together?).
References
Amazon. 2016. “Amazon Go: Frequently Asked Questions.”
(https://www.amazon.com/b?node=16008589011, accessed April 15, 2016).
Amin, S. M., and Wollenberg, B. F. 2005. “Toward a Smart Grid: Power Delivery for the 21st Century,” IEEE
Power and Energy Magazine (3:5), pp. 34–41.
Bishop, C. M. 2006. Pattern Recognition and Machine Learning, Information Science and Statistics, New
York: Springer.
Blumsack, S., and Fernandez, A. 2012. “Ready or Not, Here Comes the Smart Grid!,” Energy (37:1), pp. 61–
68.
Borgia, E. 2014. “The Internet of Things Vision: Key Features, Applications and Open Issues,” Computer
Communications (54), pp. 1–31.
Brandt, T., Feuerriegel, S., and Neumann, D. 2017. “Modeling Interferences in Information Systems Design
for Cyberphysical Systems: Insights from a Smart Grid Application,” European Journal of
Information Systems, pp. 1–14. (https://doi.org/10.1057/s41303-016-0030-1).
Chalmers, M., MacColl, I., and Bell, M. 2003. “Seamful Design: Showing the Seams in Wearable
Computing,” in Eurowearable, 2003. IEE, IET, pp. 11–16.
Chang, C.-C., and Lin, C.-J. 2011. “LIBSVM: A Library for Support Vector Machines,” ACM Transactions
on Intelligent Systems and Technology (TIST) (2:3).
Chapman, P., Clinton, J., Kerber, R., Khabaza, T., Reinartz, T., Shearer, C., and Wirth, R. 2000. “CRISP-
DM 1.0 Step-by-Step Data Mining Guide,” SPSS Inc.
Chen, T., and Guestrin, C. 2016. “XGBoost: A Scalable Tree Boosting System,” in Proceedings of the 22nd
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 785–
794.
Farhangi, H. 2010. “The Path of the Smart Grid,” IEEE Power and Energy Magazine (8:1).
Hauser, M., Griebel, M., and Thiesse, F. 2017. “A Hidden Markov Model for Distinguishing between RFID-
Tagged Objects in Adjacent Areas,” in 2017 IEEE International Conference on RFID (RFID), pp.
167–173. (https://doi.org/10.1109/RFID.2017.7945604).
Hauser, M., Zügner, D., Flath, C. M., and Thiesse, F. 2015. “Pushing the Limits of RFID: Empowering RFID-
Based Electronic Article Surveillance with Data Analytics Techniques,” in Proceedings of the Thirty
Sixth International Conference on Information Systems.
(http://aisel.aisnet.org/icis2015/proceedings/DecisionAnalytics/11/).
Hayes, R., and Blackwood, R. 2006. “Evaluating the Effects of EAS on Product Sales and Loss: Results of a
Large-Scale Field Experiment,” Security Journal (19:4), pp. 262–276.
Impinj Inc. 2017a. “SpeedwayR Installation and Operations Guide Version 5.12.0.”
(https://support.impinj.com/hc/article_attachments/115001207664/Impinj_SpeedwayR_installat
ion_and_operations_guide.pdf, accessed September 1, 2017).
Impinj Inc. 2017b. “xArray/xSpan Installation and Operations Guide Version 5.12.0.”
(https://support.impinj.com/hc/article_attachments/115001207644/Impinj_xArray_xSpan_insta
llation_and_operations_guide.pdf, accessed September 1, 2017).
Jeffery, S. R., Garofalakis, M., and Franklin, M. J. 2006. “Adaptive Cleaning for RFID Data Streams,” in
Proceedings of the 32nd International Conference on Very Large Data Bases, VLDB Endowment,
pp. 163–174.
Kang, Y., and Gershwin, S. B. 2005. “Information Inaccuracy in Inventory Systems: Stock Loss and
Stockout,” IIE Transactions (37:9), pp. 843–859.
Kourouthanassis, P., and Roussos, G. 2003. “Developing Consumer-Friendly Pervasive Retail Systems,”
IEEE Pervasive Computing (2:2), pp. 32–39. (https://doi.org/10.1109/MPRV.2003.1203751).
Designing Automated Checkout Systems for Fashion Retail Stores
Thirty Eighth International Conference on Information Systems, South Korea 2017 16
Kumar, D., Kornfield, E. M., Prater, A. C., Boyapati, S., Ren, X., and Yuan, C. 2015. Detecting Item
Interaction and Movement, U.S. Patent Application No. 13/928,345.
Lee, E. A. 2008. “Cyber Physical Systems: Design Challenges,” in Object Oriented Real-Time Distributed
Computing (ISORC), 2008 11th IEEE International Symposium on, IEEE, pp. 363–369.
Lee, I., and Sokolsky, O. 2010. “Medical Cyber Physical Systems,” in Design Automation Conference (DAC),
2010 47th ACM/IEEE, IEEE, pp. 743–748.
Litfin, T., and Wolfram, G. 2006. “New Automated Checkout Systems,” in Retailing in the 21st Century:
Current and Future Trends, M. Krafft and M. K. Mantrala (eds.), Berlin, Heidelberg: Springer Berlin
Heidelberg, pp. 143–157. (http://dx.doi.org/10.1007/3-540-28433-8_10).
Liu, H., Darabi, H., Banerjee, P., and Liu, J. 2007. “Survey of Wireless Indoor Positioning Techniques and
Systems,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)
(37:6), pp. 1067–1080. (https://doi.org/10.1109/TSMCC.2007.905750).
Manyika, J., Chui, M., Bisson, P., Woetzel, J., Dobbs, R., Bughin, J., and Aharon, D. 2015. “The Internet of
Things: Mapping the Value Beyond the Hype,” McKinsey Global Institute.
(http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/the-internet-of-
things-the-value-of-digitizing-the-physical-world).
Meuter, M. L., Ostrom, A. L., Roundtree, R. I., and Bitner, M. J. 2000. “Self-Service Technologies:
Understanding Customer Satisfaction with Technology-Based Service Encounters,” Journal of
Marketing (64:3), pp. 50–64.
National Science Foundation. 2010. “Cyber-Physical Systems (CPS).”
(https://www.nsf.gov/pubs/2010/nsf10515/nsf10515.htm, accessed April 18, 2017).
Orel, F. D., and Kara, A. 2014. “Supermarket Self-Checkout Service Quality, Customer Satisfaction, and
Loyalty: Empirical Evidence from an Emerging Market,” Journal of Retailing and Consumer
Services (21:2), pp. 118–129. (https://doi.org/http://dx.doi.org/10.1016/j.jretconser.2013.07.002).
Puerini, G. L., Kumar, D., and Kessel, S. 2015. Transitioning Items from a Materials Handling Facility, US
Patent App. 14/495,818.
Reynolds, A. P., Richards, G., Iglesia, B. de la, and Rayward-Smith, V. J. 2006. “Clustering Rules: A
Comparison of Partitioning and Hierarchical Clustering Algorithms,” Journal of Mathematical
Modelling and Algorithms (5:4), pp. 475–504. (https://doi.org/10.1007/s10852-005-9022-1).
Rousseeuw, P. J. 1987. “Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster
Analysis,” Journal of Computational and Applied Mathematics (20), pp. 53–65.
(https://doi.org/http://dx.doi.org/10.1016/0377-0427(87)90125-7).
Roussos, G., Kourouthanasis, P., Spinellis, D., Gryazin, E., Pryzbliski, M., Kalpogiannis, G., and Giaglis, G.
2003. “Systems Architecture for Pervasive Retail,” in Proceedings of the 2003 ACM Symposium on
Applied Computing, SAC ’03, New York, NY, USA: ACM, pp. 631–636.
(https://doi.org/10.1145/952532.952656).
Saha, D., and Mukherjee, A. 2003. “Pervasive Computing: A Paradigm for the 21st Century,” IEEE
Computer (36:3), pp. 25–31.
The Economist. 2016. “The Future of Personal Transport: The Driverless, Car-Sharing Road Ahead.”
(http://www.economist.com/news/business/21685459-carmakers-increasingly-fret-their-
industry-brink-huge-disruption, accessed May 6, 2017).
Weiser, M. 1991. “The Computer for the 21st Century,” Scientific American (265:3), pp. 94–104.
