Content uploaded by Kimberly Newman Frey
Author content
All content in this area was uploaded by Kimberly Newman Frey
Content may be subject to copyright.
Improve Quality of Care with Remote Activity and Fall Detection Using
Ultrasonic Sensors
Yirui Huang and Kimberly Newman, Senior Member, IEEE
Abstr act- In this paper a fall detection system is
presented that automatically detects the fall of a person
and their location using an array of ultrasonic wave
transducers connected to a field-pr ogr ammable gate
array (FPGA) processor. Experimental results are
provided on a prototype deployment installed at an
assisted living community. The system can pr ovide a
cost-effective and intelligent method to help caregivers
detect a fall quickly so that patients ar e treated in a
timely manner. In addition to room monitoring and local
aler t functions, the system incorporates a per sonal
computer and wir eless connection to enable r emote
monitoring of patient’s activity and health status.
I. INTRODUCTION
Falls are the leading cause of external injury the elderly.
In the US there are annually 1,800 falls directly resulting in
death [1] and approximately 9,500 deaths are associated with
falls each year [2]. The frequency of falls increases once
individuals have the first occurrence. This leads to a fear of
falling in many individuals that increases self-restriction of
activities [3]. In order to provide increased security and
comfort in the elderly population, automated systems with
timely reporting of fall events are critical to enable healthy
aging in place as well as limit the damage caused by injuries
during falls [4].
An intelligent yet cost-effective system that can detect a
fall in areas where they would not necessarily be wearing
clothing or desire to be monitored by a camera is the focus
for the evaluation of this system. The setting for this test case
is a bathroom in an assisted living community. The
bathroom and the resident rooms are identified as locations
where there is a high incidence of falls [5]. Identification of
the specific individual can be tied to the residential apartment
for clinical use and analysis. Currently, there are two popular
methods are used to detect a fall: machine vision and
accelerometer.
Vision tracking uses cameras at various locations and employ
either manual or automated image processing [6][7]. The
disadvantage of a vision based system is the cost of the
specialized cameras as well as the need to add privacy filters
in locations like the bathroom and bedroom. Deployment can
also be complex and costly since several devices are needed
to monitor the entire house.
Research supported by The Palisades at Broadmoor Park.
Yirui Huang is with the Electrical, Computer, and Energy Engineering
Department, University of Colorado, Boulder, CO 80309 USA (e-mail:
yirui.huang@colorado.edu).
Kimberly Newman is with the Electrical, Computer, and Energy
Engineering Department, University of Colorado, Boulder, CO 80309 USA
(e-mail: kimberly.newman@colorado.edu).
Another popular method for fall detection uses
accelerometer measurements. A fall can be defined as a
change in body orientation from upright to lying that occurs
immediately after a large negative acceleration [8]. The main
limitation of accelerometers is the inability to perform
localization and tracking of a person. Accelerometers detect
body orientation but do not have the ability to collect high-
level, context aware activity recognition based intelligence.
Another drawback for the detection of falls in a bathroom is
the need to wear the fall monitoring device. Many elderly
individuals at risk of falls do not remember to wear the
devices so their falls go undetected. In an assisted living
community, they might have to drag themselves to the pull
cord or alerting device to request help.
In this paper, a novel system is proposed using ultrasonic
waves to detect a fall. The initial pilot deployment is
described and initial measurements from a single subject are
reported. Installation is performed in the ceiling of the room
so wearable components are not required and privacy is
maintained without additional signal processing.
Preliminary measurements of the sensing mode are described
in detail by Shah, et.al, [9]. Sensing is based on the range
finder property of the ultrasonic wave which enables
determination of the latitude distribution of a certain area. A
Field Programmable Gate Array (FPGA) based processor
collects the sensor data and a mobile computer implements
an inexpensive and non-invasive monitoring system. The
Altera DE2 used in this prototype can be purchased for under
$500 and the DE0 board can be substituted for under $200.
The computer can be purchased for $250, sensors for $28.00
each unit, and the remainder of the system components are an
A/D converter and Ethernet cables for the wiring. Total cost
for deployment of 8 monitoring devices in a bathroom,
excluding labor, is possible for approximately $750. The
components of this system are shown in fig. 1.
II. SYSTEM ARCHITECTURE
The system is divided into two main components: the
hardware, which is responsible for collection and processing
sensor data, and the software, which is responsible for
sending out alerts and notifications.
The hardware is an array of ultrasonic sensors used to
measure distance between the sensor and an object within
the shape of the ultrasonic wave up to maximum distance of
254 inches. The beam width changes with this distance as
demonstrated previously [9]. Sensors are installed on a flat
surface so the ping has a known trajectory without
interference in the path. Only one transmitter is active at a
time to prevent overlapping of signals so that reflections can
be isolated. When an object is detected in the field of the
active sensor it can locate the distance within one inch of
34th Annual International Conference of the IEEE EMBS
San Diego, California USA, 28 August - 1 September, 2012
5854978-1-4577-1787-1/12/$26.00 ©2012 IEEE
resolution. This approach enables a plot of the latitude
distribution for the plane facing the sensor surface.
Figure 1: Fall Monitoring Components
Abnormal activity is detected by comparison with the
default scenario in the room as well as the actions of the
previous set of measurements. The dimensions of the room
constrain the speed at which each position is checked,
however, sensors can be triggered at a rate of 50ms so for a
deployment in a 10 foot by 10 foot room with ceiling height
of 10 feet, the density of sensors would be 4 and the
maximum amount of time between polling of an individual
sensor would be 50 ms. As shown in fig. 2, the default
value for a room is on the left assuming furniture is not
present. Furniture is mapped as part of the initialization
process and is included as a background measurement. If
the furniture is moved during system operation, the software
can detect this background change but it is difficult in the
current version of the deployment to differentiate between
individuals and objects. Methods to improve this sensitivity
are discussed in the results section of the paper.
When a person walks under the beam, the return reading
is compared to the default and assumptions are made as to
the pose of the individual. The range of expected height
values for walking are calibrated to the resident during
installation. Additional determination of the seated height is
performed during installation. Falls are detected by a rapid
transition from the standing or seated position to the ground.
The software shown in the upper right of fig. 1, transmits
the sensor responses returned by the FPGA to a netbook
using an R232 cable. A Java program continuously reads the
sensor responses from the serial port and sends out
notifications every 3 minutes based on the sensor response
status (“OK” or “EMR”). If an emergency (EMR) is
detected, the EMR signal is sent out immediately to the
email of individuals registered in the setup. Testing of these
alerts can also be triggered manually to ensure the system is
communicating.
Figure 2: Pose recognition
III. FALL DETECTION ALGORITHM
The algorithm includes two parts: initialization and
decision-making. The initialization procedure aims to
establish an idle condition of the measured plane. The flow
chart of the operation of this algorithm is provided in fig. 3.
An idle condition is assumed if a sensor has a constant
reading for 60 seconds. There are four functions used for the
initialization procedure. The input function reads the
ultrasonic sensors measured result. The save function places
the value from the sensor into the base value variable. A five
second delay between sensor readings is implemented with
the delay function. This is a rather crude averaging filter
similar to mechanical switch debouncing to reduce the noise
in the input. The compare function is used after a second
input reading is taken to see if the initial value and the final
value match so that false readings are avoided.
Figure 3: Initialization Procedure
After the initialization routine is performed to gain a
baseline measurement of the room, a second algorithm is
used for regular monitoring of the sensor values as shown in
fig. 4. The decision-making procedure is able to detect the
sign of a fall. This algorithm compares the sensor’s present
result with the base value from the initialization routine with
two time parameters. The first time parameter counts the
time elapsed after an abnormal activity. A second time
parameter is used to eliminate the pseudo-sign of fall. There
is a looping function used for the decision-making
procedure. The system sequentially reads each sensor and
uses a compare function to detect the sign of fall. The
compare function checks whether the present reading from a
sensor is consistent with the saved base reading. The system
will recheck the same sensor to eliminate the pseudo-sign of
5855
fall after a five second delay in case of an inconsistent result
returned by the compare function. Further, the system will
issue an emergency signal if a different reading at a sensor
lasts for more than one minute.
Figure 4: Decision-Making Procedure
IV. Eldercare Monitoring Deployment
After characterization of the system in the controlled
environment of a lab, an IRB-approved informed consent
study was initiated at the Palisades assisted living
community in Colorado Springs. A subject in the assisted
care wing was recruited and the system was installed in the
bathroom. For the initial trial, only 8 ultrasonic sensors are
used and the complete installation limited to the bathroom –
an area in which elderly people commonly fall. Three
sensors are placed in the shower and the five remaining
sensors are placed around the rest of the bathroom to provide
sufficient fall coverage. The installation is shown in the
images of fig 5.
Installation of the system required a significant amount
of time and testing but this was expected for the pilot
version. Sensors are mounted on the ceiling as shown in
5(a) and the cables are run to the FPGA and netbook which
is placed on the shelf as shown in 5(b). Proper placement
and leveling of the sensors as well as calibration for the user
was the majority of the time commitment.
Several site visits were also required during the first
three months for the fine tuning of the system to allow
remote access and tweak the fall detection algorithms to
avoid false positives. Additionally, a user manual was
created to provide training to the staff on the use of the
system. A screen shot of the user interface is shown in fig. 6.
This interaction is still in progress but the system operated
for several weeks at a time for the remainder of the six
month trial. Late night IT maintenance caused some
problems with stability and required resetting of the Twitter
interface remotely. There was also the need for
reinstallation of one of the sensors on the ceiling of the
bathroom since it became unattached due to the humidity in
the environment.
(a) Sensor Installation
(b) Netbook Placement
Figure 5: Bathroom Installation at Palisades
Figure 6: Health Monitor User Interface
Currently, the system sends out automated notifications
using both Twitter and email, allowing alerts through
different channels of communication, quickly, and to
multiple recipients. Data logging is not currently performed
so the height and specific location of the falls are not
reported. There are plans are to incorporate this into the data
transfer but the first step is to stabilize the system in the field
and ensure the Java and Twitter interfaces are capable of
reporting activity.
The system updates the status of the subject to a
specified Twitter account which is available to all
caregivers. This data is composed of OK pings every five
minutes. When the EMR occurs, an email is generated
immediately as shown in fig. 7.
Figure 7: Twitter and Email Sample of the System
Verification of the detection of the fall is collected by
direct phone contact with the Palisades to determine if the
system is operating properly or if it is sending a false alarm.
Over the course of the trial, there was a weekend where the
system sent out several alarms. Three alarms were sent out
over the course of an evening and then the system was reset
5856
using the logmein tool. Emergency messages continued that
weekend around noon the next day and also late in the
evening in the month of August. It was discovered there
was a glitch in the software interface between the Java code
and the twitter account that was causing these false alarms
when the internet connection is lost. Modifications were
made to account for the evening IT maintenance so that
signals could be buffered and the system would reset after
the maintenance was performed. In December there was
another long interruption to the system over a weekend and,
an actual fall was missed. After the event in December, the
system was reset and the monitoring was in operation
through January. No falls were detected during this time
frame and the system remained operational.
V. CONCLUSION
After a six month trial in the Palisades Assisted Living
community, the system demonstrated the capability to detect
falls. This length of time was required for the first
deployment to fine tune the software and remove the
occurrence of false positives. In the future, deployments in
additional rooms are possible and should require less time
for operational stability based on the knowledge gained in
this pilot trial.
The system is capable of monitoring and recording the
activity of a person using ultrasonic range finding. The
system is able to collect the data of a person’s daily activity
and develop his (or her) activity pattern using a sophisticated
algorithm and the signal processing capability provided by
an FPGA processor. Daily activity patterns can be stored in
future versions of this system to help physicians and
caregivers determine activities that lead to a fall and detect
the fall in a timely manner to reduce risk of further injury.
There are additional refinements required before the
system is ready to expand into additional rooms. The ability
to store data is beneficial so that activity patterns can be
detected and locations of falls can be identified. The
implementation of the sensors on the ceiling could be
improved by incorporating a wireless interface to eliminate
the use of the Ethernet cables currently connecting the
sensors to the FPGA. Additionally, the portability of the
remote monitoring software could be enhanced by creating
an interactive web based interface. The new interface would
free the users from their ordinary workspaces and give more
flexibility on choice of terminal devices. Users can
download the stored data from the system directly from the
web.
Another issue that was not expected or factored into the
pilot deployment was the use of a walker by the resident.
Readings of the height of the walker made it difficult to
distinguish where the person was in relation to this device
since it was moving in the space. For example, if the person
fell near the walker, would the height change be detected
with this in the field. In addition to the sensors placed on the
ceiling, sensors could be placed on the walls so that
measurements can be made in the horizontal and vertical
plane to isolate a person from a walker. Other single-point
sensing modes could be employed that do not violate privacy
of the resident. A combination temperature and humidity
sensor could distinguish the heat signature of a person and
from a device.
Ultrasonic technology offers promising potential in the
field of activity monitoring. This project hopes to provide
additional insight into the use of ultrasonic activity monitors
and establish a solid foundation on which future
developments can be made. With further improvements and
additional research, ultrasonic activity monitoring systems
have the potential to work both effectively and practically in
reducing the number of hospitalizations and deaths caused
by elderly falls. In time, such systems should be capable of
being not only reliable and accurate in detecting falls, but
also affordable and user friendly.
VI. ACKNOWLEDGEMENT
We would like to thank Maulik Kapuria, Niket Shah,
Seyitriza Tigrek, Tim Ikenouye, Willie Long and Saurabh
Goel for their efforts on the creation of the initial prototypes
of this system. We are also grateful to Professor Frank
Barnes, Karen Siegel, and Professor Sara Qualls for their
expertise and critique of this manuscript.
VII. REFERENCES
[1] L. K. Dial, Conditions of aging. Baltimore: Lippincott
Williams & Wilkins, 1999, pp.119–26.
[2] M.A. Forciea, R. Lavizzo- Mourey. E.P. Schwab,
Geriatric secrets. 2nd ed Hanley & Belfus, 1996, pp. 110–5.
[3] G. F. Fuller, “Falls in the Elderly,” American Family
Physician, vol. 61, no. 7, 2000, pp. 2159-2168
[4] Centers for Disease Control and Prevention, National
Center for Injury Prevention and Control.
Web–based Injury
Statistics Query and Reporting System (WISQARS)
[online]. Accessed November 30, 2010.
[5] K. Rapp, C. Becker, I.D. Cameron, H.H. Konig, G.
Buchele, “Epidemiology of falls in residential aged care:
analysis of more than 70,000 falls from residents in nursing
homes,” Journal of the American Medical Directors
Association, vol. 3, no. 2, pp. 187.e1-187.e6.
[6] W. Hu, T. Tan, L. Wang, S. Maybank, “A Survey on
Visual Surveillance of Object Motion and Behaviors,” IEEE
Transactions on Systems, Man, and Cybernetics-Part C:
Applications and Reviews, vol.34, no. 3, 2004, pp.334-352.
[7] G. Diraco, A. Leone, P. Siciliano, “An Active Vision
System for Fall Detection and Posture Recognition in
Elderly Healthcare,” Design, Automation, and Test in
Europe Conference and Exhibition, 2010, pp.1536-1541.
[8] G. Williams, K. Doughty, K. Cameron and D.A.
Bradley, “A smartfall and activity monitor for telecare
applications,” Proc. 20
th
Annual Int. Conf of the IEEE
Engineering in Medicine and Biology Society, vol.3,
1998, pp.1151-1154. [9] N. Shah, M. Kapuria, K.E.
Newman, Chapter 13: Embedded Activity Monitoring
Methods, Activity Recognition in Pervasive Intelligent
Environments, Atlantis Ambient and Pervasive Intelligence,
vol. 4, 2011, pp. 291-311.
5857
