Content uploaded by Jun Hyung Jo
Author content
All content in this area was uploaded by Jun Hyung Jo on Jun 26, 2014
Content may be subject to copyright.
Heart Rate Prediction Based on Physical Activity using
Feedforward Neural Network
Ming Yuchi
School of Life Science and Technology
Huazhong University of Science and Technology
Wuhan, Hubei, China
m.yuchi@gmail.com
Jun Jo
School of Information and Communication Technology
Griffith University
Queensland, Australia
j.jo@griffith.edu.au
Abstract
The technique of combining heart rate (HR) and physi-
cal activity (PA) has been adopted in a number of research
areas, such as energy expenditure measurement, autonomic
nervous system assessment, sports research, etc. However,
there have been few studies on the direct relationship be-
tween HR and PA. This paper proposes a HR prediction
model based on the relationship between HR and PA. The
predictor has the potential to be used in various areas, such
as: cardiopathy research and diagnosis, heart attack warn-
ing indicator, sports capability measure and mental activ-
ity evaluation. The method has the following steps: first,
the recorded HR and PA signals are preprocessed as two
synchronized time sequences: HR(n) and PA(n).Thein-
puts of the predictor are HR(n) and PA(n) in the cur-
rent time step, and the output is the predicted sequence
HR(n +1)in the next time step. The Feedforward Neu-
ral Network (FFNN) was chosen as the mathematical model
of the predictor. Experiments was conducted based on the
real-life signals from a healthy male. A set of 90 minute
signals were collected. One half of the signal set was used
to train the FFNN and the other half to validate the train-
ing. The mean absolute error of the predicted heart rate
was restricted inside 5. The result shows the potential of
the proposed method.
1 Introduction
The heart rate (HR) is generally measured as a series of
time intervals (the so-called RR intervals) between the heart
cycles that are obtained from the electrocardiogram (ECG)
[1]. Analysis on HR has become a popular noninvasive tool
for the studies on cardiopathy and exercise physiology.
One limitation associated with HR monitoring technique
is that it is difficult to identify whether the HR increase is
due to physical activity (PA) or mental activity [2, 3], es-
pecially when the increase in heart rate is modest. One
possible solution to this problem is to incorporate the PA
signals into the investigation scope. Research projects and
applications that combine HR and PA signals include: en-
ergy expenditure measurement [4] - [6], autonomic nervous
system assessment [7] - [11], sports research [12] - [14].
Most of the above works utilize the HR and PA as two
parallel inputs, and the output of the system is energy ex-
penditure, oxygen consumption or nervous system routine.
There have been only a few studies looking into the direct
relationship between HR and PA. Pawar el al. [15] pre-
sented one body movement activity detection system which
is based on ECG signal, but not HR. Meijer el al. [3] built a
linear-type relationship between the HR and the body move-
ment. However, the experiments were implemented within
specific conditions and the body movement was recorded as
the counted number of activities, which could not appropri-
ately reflect the actual PA.
The main purpose of this paper is to build a HR predic-
tion model, which is based on real-life HR and PA (3-D
acceleration) signals. This model can be further developed
International Conference on Convergence and Hybrid Information Technology 2008
978-0-7695-3328-5/08 $25.00 © 2008 IEEE
DOI 10.1109/ICHIT.22
344
International Conference on Convergence and Hybrid Information Technology 2008
978-0-7695-3328-5/08 $25.00 © 2008 IEEE
DOI 10.1109/ICHIT.2008.175
344
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on July 23, 2009 at 19:25 from IEEE Xplore. Restrictions apply.
to a HR abnormality detection system. For the experiment,
the subject was equipped with the portable HR and PA mon-
itor, then proceeded to perform normal daily activities with-
out any special routine or restriction. The recorded HR and
PA signals were preprocessed to produce two synchronized
time sequences: HR(n) and PA(n). The inputs of the pre-
dictor are HR(n) and PA(n) in the current time step, and
the output is the predicted sequence HR(n +1)in the next
time step.
Considering that all of the signals are non-constrained
and real-time data, the predictor has the potential to be used
in various areas, such as: cardiopathy research and diagno-
sis, heart attack warning indicator, sports capability mea-
sure and mental activity evaluation. One of the possible
practical application is to integrate the predictor with the
portable HR device to monitor asymptomatic HR sudden
change, which is common for early-stage heart disease pa-
tient. The predictor compares the real HR and predicted one
every time step, if the difference exceeds one predefined tol-
erance value, the device can warn the wearer or mark this
part of ECG signal for diagnose reference.
The relationship between HR and PA is affected by many
factors, such as age, sex, mental stress, ambient tempera-
ture hydration, etc. It is difficult to identify the direct rule
behind the relationship. For this reason, we adopted feed-
forward neural network (FFNN) [16] - [18] as the mathe-
matical model for the predictor, based on its intrinsic non-
linearity and computational simplicity.
This paper is organized as follows: firstly, the proposed
method is presented in Section 2, which includes the intro-
duction to the entire system, the signal recorder, the sig-
nal preprocessing and the FFNN. In Section 3, the predic-
tor is tested with the signals obtained from one 33 year old
healthy male. Finally, concluding remarks and discussions
follow in Section 4.
2 The Research Method
2.1 HR Prediction Model
To investigate the relationship between the HR and PA,
we need simultaneously-recorded HR and PA signals. One
portable HR and PA monitor from Alive Technologies was
used here. The monitor measures and records the wearer’s
ECG and PA (3-D acceleration) signals and determines the
HR from the ECG in real-time. The left part of Fig. 1 shows
the subject (user) wearing the monitor. The specification of
the monitor will be described in Section 2.2.
The middle part of Fig. 1 is the preprocessor which
converts the acquired HR (hr(m)) and acceleration signals
(ac
x
(l), ac
y
(l), ac
z
(l)) into usable format. The outputs
of the preprocessor include two synchronized sequences
HR(n) and PA(n), which are forwarded to the FFNN as
Table 1. Data Specification of Alive Heart
Monitor
Signal ECG Accelerometer
Channels/Axis Single Channel 3 Axes
Resolution 8 bits 8 bits
Sampling Rate 300 samples/sec 75 samples/sec
Dynamic Range −2.66mV ∼ 2.66mV −2.7g ∼ 2.7g
Bandwidth 0.5Hz ∼ 90Hz 0Hz ∼ 20Hz
inputs. The output of the neural network is
HR(n +1),
which is the predicted HR on next time step.
a
b
b
Preprocessor
Heart Rate hr (m)
Physical Activity
ac
x
(l), ac
y
(l), ac
z
(l)
Neural
Network
HR (n)
PA (n)
HR (n+1)
Z
HR (n+1)
+
−
e (n+1)
~
a
b
b
Preprocessor
Heart Rate hr (m)
Physical Activity
ac
x
(l), ac
y
(l), ac
z
(l)
Neural
Network
HR (n)
PA (n)
HR (n+1)
Z
HR (n+1)
+
−
e (n+1)
~
Figure 1. The block diagram of the whole sys-
tem. a: a monitor which senses and records
heart rates and physical activities; b: Elec-
trodes.
2.2 Heart Rate and Physical Activity
Recorder
Many studies on HR are based on the experimental
data gathered in specific conditions and/or environments,
whereas, this research was conducted with the data col-
lected from normal daily activities, without requiring any
pre-planned routine. Consequently, we need one portable
device, which can monitor and record the HR and PA sig-
nals simultaneously for a period of time with relatively high
accuracy.
According to the device requirements, one commercial
product Alive Heart Monitor (AHM) is chosen for our ex-
periments. Owing to its small size and light weight, the
AHM can be worn comfortably during normal daily activ-
ities. The monitor gathers the single-channel ECG signal
from a pair of electrodes attached at certain positions of the
subjects’s skin and 3-D physical activity (acceleration) sig-
nals from one build-in 3 axis accelerometer. The collected
data can be saved in an internal SD memory card or trans-
mitted to PC, smartphone or PDA using Bluetooth in real
time. The AHM uses a rechargeable Lithium-ion battery
which provides about four days of data saving or three days
of wireless-transmission, continuously. The data specifica-
tion of the AHM is shown in Table 1.
345345
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on July 23, 2009 at 19:25 from IEEE Xplore. Restrictions apply.
Table 2. HR(n) and PA(n) Values of Fig. 3
n HR(n)(bpm) PA(n)(g)
1 122.5 0.62682
2 121.5 0.66212
3 122.75 0.67144
The HR reading is generated on RR intervals of collected
ECG signals. The sampling rate of HR is 1 samples/sec.
Each HR is worked out as the average length of nine se-
quential RR intervals to reduce the influences of false or
missing beat detections and ectopic beats.
Two examples of recorded signals are shown in Fig. 2,
which includes the data of ECG, HR, acceleration of x, y
and z axes. Since the orientation of the axes may change
along with the subject’s movement, a certain amount of off-
set is added to each acceleration signal according to the ori-
entation of the axis, which may help identify the body an-
gles or the physical status of the subject.
While keeping a crouching posture in Fig. 2(a), the sub-
ject performs a jumping action between 1.5s and 3s in Fig.
2(b). It can be found that the jumping movement created
some noises to the ECG signals, which may influence the
accuracy of HR calculation.
2.3 Signal Preprocessing
The sampling rates of HR and acceleration are set dif-
ferently in the AHM (1 samples/sec and 75 samples/sec, re-
spectively) even though the inputs of the neural network are
required to be sequences with same sampling rate. Here, we
convert hr(m) and ac
x
(l), ac
y
(l), ac
z
(l) into two synchro-
nized sequences HR(n) and PA(n) through a processing
period τ.
Assume the whole recording period is T , the recorded
data on each signal channel are evenly divided into N seg-
ments, each segment has the length of τ,
N = floor(T/τ), (1)
where floor(x) rounds x to the nearest integers towards
minus infinity. The left part of Fig. 3 shows an example
with T =12s and τ =4s. The recorded data are divided
into N =12/4=3segments on each channel. Each HR
segment has N
hr
samples,
N
hr
= samplingRate
hr
× τ; (2)
and each acceleration segment has N
ac
samples,
N
ac
= samplingRate
ac
× τ. (3)
When τ =4s, HR segment has 1 samples/s ×4s =4
samples, and each acceleration segment has 75 samples/s
×4s = 300 samples.
0 1 2 3 4 5
−2
0
2
Sit
ECG(mV )
0 1 2 3 4 5
80
100
120
hr(bpm)
0 1 2 3 4 5
−2
0
2
ac
x
(g)
0 1 2 3 4 5
−2
0
2
ac
y
(g)
0 1 2 3 4 5
−2
0
2
ac
z
(g)
tim e(s)
(a)
0 1 2 3 4 5
−2
0
2
Jump
ECG(mV )
0 1 2 3 4 5
80
100
120
hr(bpm)
0 1 2 3 4 5
−2
0
2
ac
x
(g)
0 1 2 3 4 5
−2
0
2
ac
y
(g)
0 1 2 3 4 5
−2
0
2
ac
z
(g)
tim e(s)
(b)
Figure 2. Examples of recorded AHM data:
ECG, HR, Acceleration in x-axis, y-axis, z-
axis. (a) Subject sat for five seconds; (b) Sub-
ject performed one jump action in five sec-
onds.
346346
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on July 23, 2009 at 19:25 from IEEE Xplore. Restrictions apply.
0 2 4 6 8 10 12
110
120
130
τ =4s
hr(bpm)
0 2 4 6 8 10 12
−2
−1
0
ac
x
(g)
0 2 4 6 8 10 12
−1
0
1
ac
y
(g)
0 2 4 6 8 10 12
−1
0
1
ac
z
(g)
time(s)
PA(n):PA(1),PA(2),PA(3)
HR(n):HR(1),HR(2),HR(3)
Figure 3. An example of Data preprocessing:
T =12s and τ =4s. 3 segments are formed
on each channel. Then, these segments are
converted into two synchronized sequences,
HR(n) and PA(n).
Then, the nth (n =1,...,N) hr segment is converted
into HR(n),andthenth ac
x
, ac
y
, ac
z
segments are con-
verted into PA(n), through the following functions,
HR(n)=
n∗N
hr
m=(n−1)∗N
hr
+1
hr(m)
N
hr
, (4)
PA(n)=
n∗N
ac
−1
l=(n−1)∗N
ac
+1
{|ac
x
(l)−ac
x
(l+1)|+|ac
y
(l)−ac
y
(l+1)|+|ac
z
(l)−ac
z
(l+1)|}
N
ac
.
(5)
HR(n) is the average heart rate of nth segment. PA(n)
is also worked out as an average value. However, instead
of the acceleration signals being directly used, the absolute
difference value of adjacent acceleration signals is adopted
to calculate PA(n). This reflects the PA change between
adjacent time steps, and eliminates the influences of the off-
set added on the acceleration signals. The right part of Fig.
3 shows the two sequences HR(n) and PA(n), and Table
2 lists the corresponding values.
It should be noted that the function of τ is not only
to synchronize the inputs to neural network, but also to
help stabilize the prediction accuracy through averaging the
noises. This works well, especially when some signals have
high noises, e.g., the jumping action in Fig. 2(b).
2.4 Feed Forward Neural Network
In this work, there exists two factors which increase the
difficulty of the prediction. The first factor is that the sub-
ject performs normal daily activities. The consequence is
that the recorded HR is influenced by different aspects, such
as, the subject’s body condition, mood and surrounding en-
vironment. The second factor is that the data is collected
from one portable monitor. The accuracy and precision of
the device may be limited compared to the equipment in a
hospital laboratory.
These factors adds uncertainties to the experiments. In
fact, HR(n) and PA(n), HR(n) and HR(n +1)show
nonlinear relationships in the data set obtained from the
AHM, especially when τ is a relatively small value. There-
fore, a mathematical method aiming at nonlinear prediction
is needed. FFNN appears to be a good candidate [19, 20].
With a certain structure, multi-layer FFNN can be used as a
general function approximator [21] - [23].
A FFNN [16] - [18] is a biologically inspired classifi-
cation algorithm. It consist of a (possibly large) number
of simple neuron-like processing units, organized in layers.
Every unit in a layer is connected with all the units in the
previous layer. These connections are not all equal, each
connection may have a different strength or weight. The
weights on these connections encode the knowledge of a
network. Often the units in a neural network are also called
neurons(nodes).
Data enters at the inputs and passes through the network,
layer by layer, until it arrives at the outputs. During normal
operation, there is no feedback between layers. This is why
they are called feedforward neural networks.
Without needing any mathematical knowledge between
the input and output, the FFNN is trained based on compar-
isons of the output and the target, until the network matches
the target (Fig. 1).
3 Pilot Experiment
3.1 Experiment Specifications
In this paper, the subject was chosen as a 33 years male
with no record of heart disease. The recording time period
was 90 minutes. During this continuous period, the sub-
ject wore an AHM and performed the following activities:
sitting and reading on the sofa, walking to the bus station,
running to catch a bus, sitting in the bus, walking to the
office, and other normal actions in his office.
The recorded signals were evenly separated as two parts.
The first 45 minute signals were adopted as the training
set, which was used to train the FFNN. The remaining 45
minute signals were for the test set, which was used to val-
idate the trained neural network. The preprocessing param-
eter τ can be set with different values based on the user’s
practical desire. Here, τ was set to be 30s for experiment
test. Various values of τ will be tested in the future work.
Therefore, for both the training and test sets, N =90in
Fig. 4 and Fig. 5 elucidate the corresponding HR(n) and
PA(n) of the training set and test test.
The Neural Network Toolbox of the Matlab 7 was chosen
to generate and train the neural network. Two-layer FFNN
was selected as the predictor for this experiment. The two
347347
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on July 23, 2009 at 19:25 from IEEE Xplore. Restrictions apply.
0 10 20 30 40 50 60 70 80 90
70
80
90
100
110
120
Training Set
HR(n)
0 10 20 30 40 50 60 70 80 90
0
2
4
6
8
10
PA(n)
n
Figure 4. Training set for neural network train-
ing, T =45min, τ =30s, N =90.
inputs of the FFNN were HR(n) and PA(n). The out-
put layer (the last layer) had one neuron,
HR(n +1),the
predicted HR of the next time step. The hidden layer (first
layer) had 50 neurons. The number of hidden neurons is
selected based on test-and-trial method. Fig. 6 shows the
structure of the FFNN used in this paper.
Normally, the FFNN is trained with a backpropaga-
tion method, which includes many variations. Here, the
Levenberg-Marquardt backpropagation method [24, 25]
was adopted, based on its good performance and fast train-
ing speed for moderate-sized FFNN [26]. The network was
trained for 200 generations on the training set.
3.2 Experimental Results
The performance of the neural network predictor on the
training set and test set is shown in Fig. 7 and Fig. 8. To
make a clear identification, the predicted
HR(n +1)is de-
noted with a dashed line, while the original HR(n +1)is
represented by a unbroken line. The figures indicate that the
HR(n +1)follow the variance of HR(n +1)on both the
training set and test set after training.
The residual errors between the the HR(n +1)and
HR(n +1)are also shown in Fig. 7 and Fig. 8. The cor-
responding mean absolute error (MAE) and the variance of
the error are listed in Table 3. Considering that the experi-
ment was worked on real-life data and the prediction inter-
val was only 30s, the MAE on both training and test sets is
acceptable. However, the variance of the error is still rela-
tively large. In Fig. 7 and Fig. 8, some residual errors are as
big as 15, although most of the residual errors are smaller
0 10 20 30 40 50 60 70 80 90
70
80
90
100
110
120
Test Set
HR(n)
0 10 20 30 40 50 60 70 80 90
0
2
4
6
8
10
PA(n)
n
Figure 5. Test set for neural network valida-
tion, T =45min, τ =30s, N =90.
...
Inputs Hidden Layer Output Layer
HR(n)
PA(n)
HR(n+1)
~
Figure 6. Two-layer FFNN structure.
than 5. Reducing the variance value to maintain the con-
sistency of the prediction remains to be an objective of our
future research.
4 Conclusion and Discussion
In this paper, a HR predictor based on PA was proposed.
The predictor has the potential to be used in various areas,
such as: cardiopathy research and diagnosis, heart attack
warning indicator, sports capability measure and mental ac-
tivity evaluation. FFNN was adopted as the mathematical
model of the predictor. Experiments was conducted on 90
minutes of real life data were collected from one 33 year old
healthy male who wore the heart monitor, AHM. The pre-
diction was performed every 30 second. The result showed
the potential of the predictor with the results close to the ac-
tual data. The mean absolute error could be restricted within
a small range. The consistency of the prediction needs be
348348
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on July 23, 2009 at 19:25 from IEEE Xplore. Restrictions apply.
0 10 20 30 40 50 60 70 80 90
70
80
90
100
110
120
Training Set
0 10 20 30 40 50 60 70 80 90
−20
0
20
Residual Error
HR(n +1)
HR(n +1)
Figure 7. Feedforward neural network predic-
tor performance on the training set: HR(n+1)
and
HR(n +1), and the corresponding resid-
ual error.
Table 3. MAE and Variance of the prediction
error
Data Set MAE Variance
Training Set 3.12 16.62
Test Set 3.31 18.68
improved and will be addressed in the future work.
To validate the effective of the proposed method and im-
prove the neural network performance, further and deeper
investigations are needed. Firstly, many and various sub-
jects are needed. The current experiment was tested on one
healthy male. Data from subjects of varying age, gender
and health level should be tested. With assistance from the
medical profession, the experiment could be implemented
with hospital patients. Secondly, more tests on different
system parameters and structures are needed. The possible
varying factors include: prediction interval and total time
length, data structure, neural network structure and training
method.
References
[1] T. F. ESC/ASPE, “Heart rate variability: Standards of
measurement, phisiological interpretation, and clini-
cal use,” Circulation, vol. 93, pp. 1043–1065, 1996.
[2] W. L. Haskell, M. C. Yee, A. Evans, and P. J. Irby, “Si-
multaneous measurement of heart rate and body mo-
0 10 20 30 40 50 60 70 80 90
70
80
90
100
110
120
Test Set
0 10 20 30 40 50 60 70 80 90
−20
0
20
Residual Error
HR(n +1)
HR(n +1)
Figure 8. Feedforward neural network predic-
tor performance on the test set: HR(n +1)
and
HR(n +1), and the corresponding resid-
ual error.
tion to quantitate physical activity,” Med. Sci. Sports
Exerc., vol. 25, pp. 109–115, 1993.
[3] G. A. Meijer, K. R. Websterp, H. Koper, and F. ten
Hoor, “Assessment of energy expenditure by record-
ing heart rate and body acceleration,” Medicine and
Science in Sports and Exercise, vol. 21, pp. 343–347,
1989.
[4] K. Rennie, T. Rowsell, S. A. Jebb, D. Holburn, and
N. J. Wareham, “A combined heart rate and move-
ment sensor: proof of concept and preliminary test-
ing study,” European Journal of Clinical Nutrition,
vol. 54, pp. 409–414, 2000.
[5] S. Brage, N. Brage, P. W. Franks, U. Ekelund, M.-Y.
Wong, L. B. Andersen, K. Froberg, and N. J. Ware-
ham, “Branched equation modeling of simultaneous
accelerometry and heart rate monitoring improves es-
timate of directly measured physical activity energy
expenditure,” Journal of Appllied Physiology, vol. 96,
pp. 343–351, 2004.
[6] J. K. Moon and N. F. Butte, “Combined heart rate and
activity improve estimates of oxygen consumption and
carbon dioxide production rates,” Journal of Appliied
Physiology, vol. 81, pp. 1754–1761, 1996.
[7] H. L. Chan, M. A. Lin, P. K. Chao, and C. H. Lin,
“Correlates of the shift in heart rate variability with
postures and walking by time frequency analysis,”
349349
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on July 23, 2009 at 19:25 from IEEE Xplore. Restrictions apply.
Computer Methods and Programs in Biomedicine,
vol. 86, pp. 124–130, 2007.
[8] H. L. Chan, S. C. Fang, Y. L. Ko, M. A. Lin, H. H.
Huang, and C. Lin, “Heart rate variability characteri-
zation in daily physical activity using wavelet analysis
and multi-layer fuzzy activity clustering,” IEEE Trans.
Biomed. Eng., vol. 53, pp. 133–139, 2006.
[9] R. Perini and A. Veicsteinas, “Heart rate variability
and autonomic activity at rest and during exercise in
various physiologicalconditions,” Eur. J. Appl. Phys-
iol., vol. 90, pp. 317–325, 2003.
[10] R. Perini, S. Milesi, N. M. Fisher, and D. R. Pender-
gast, “A. veicsteinas, heart rate variability during dy-
namic exercise in elderly males and females,” Eur. J.
Appl. Physiol., vol. 82, pp. 8–15, 2000.
[11] L. Mourot, M. Bouhaddi, S. Perrey, J. D. Rouillon,
and J. Regnard, “Quantitative poincar´e plot analysis
of heart rate variability: effect of endurance training,”
Eur. J. Appl. Physiol., vol. 91, pp. 79–87, 2004.
[12] L. H. Epstein, R. A. Paluch, L. E. Kalakanis, G. S.
Goldfield, F. J. Cerny, and J. N. Roemmich, “How
much activity do youth get? a quantitative review
of heart-rate measured activity,” Pediatrics, vol. 108,
no. 3, pp. 1–10, 2001.
[13] W. Wang, J. R. Wei, D. C. Zhang, S. Z. Xiao, and F. C.
Wang, “A study on 6-minute walk test incorporating
cardiac contractility and heart rate change measure-
ments,” Chinese Medical Equipment Journal, vol. 24,
no. 6, pp. 16–18, 2003.
[14] Y. S. Qin, M. Zhang, Z. G. Hao, and L. X. Song, “The
use of the heart rate in training,” Chinese Journal of
Clinical Rehabilitation, vol. 7, no. 6, pp. 952–953,
2003.
[15] T. Pawar, S. Chaudhuri, and S. P. Duttagupta, “Body
movement activity recognition for ambulatory cardiac
monitoring,” IEEE Trans. Biomed. Eng., vol. 54, no. 5,
pp. 874–882, 2007.
[16] M. T. Hagan, H. B. Demuth, and M. H. Beale, Neu-
ral Network Design. Boston: MA: PWS Publishing,
1996.
[17] A. J. Annema, Feed-Forward Neural Networks: Vec-
tor Decomposition Analysis, Modelling and Apalog
Implementation. Boston: Springer, 1995.
[18] S. Haykin, Neural Networks - A Comprehensive Foun-
dation. New York: Macmillan College Publication,
1994.
[19] W. Y. Huang and R. P. Lippmann, “Comparisons be-
tween neural net and conventional classifiers,” Lincoln
Laboratory, MIT, Tech. Rep., 1987.
[20] ——, “Neural net and traditional classifiers,” Lincoln
Laboratory, MIT, Tech. Rep., 1987.
[21] K. Hornik, M. Stinchcombe, and H. White, “Multi-
layer feedforward networks are universal approxima-
tors,” Neural Networks, vol. 2, pp. 359–366, 1989.
[22] E. K. Blum and L. K. Li, “Approximation theory and
feedforward networks,” Neural Networks, vol. 4, pp.
511–515, 1991.
[23] F. Scarselli and A. C. Tsoi, “Universal approximation
using feedforward neural networks: a survey of some
existing methods, and some new results,” Neural Net-
works, vol. 11, pp. 15–37, 1998.
[24] K. Levenberg, “A method for the solution of certain
non-linear problems in least squares,” Quart. Appl.
Math., vol. 2, pp. 164–168, 1944.
[25] D. Marquardt, “An algorithm for least-squares estima-
tion of nonlinear parameters,” SIAM J. Appl. Math.,
vol. 11, pp. 431–441, 1963.
[26] M. T. Hagan and M. Menhaj, “Training feedforward
networks with the marquardt algorithm,”
IEEE Trans-
actions on Neural Networks, vol. 5, no. 6, pp. 989–
993, 1994.
350350
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on July 23, 2009 at 19:25 from IEEE Xplore. Restrictions apply.
