HIGH RESOLUTION NEARLY-ML ESTIMATION OF SINUSOIDS IN NOISE USING A FAST FREQUENCY DOMAIN APPROACH

Abstract

Estimating the frequencies, amplitudes and phases of sinusoids in noise is a problem which arises in many applications. The aim of the methods in this paper is to achieve computational eciency and near-ML perfor-mance (i.e. low bias, variance and threshold SNR), in problems such as vibration or audio analysis where the number of tones may be large (e.g. > 20). An approach has recently been published for resolved tones [4]. This paper extends that frequency domain approach t o the high-resolution problem.

Full text

HIGH RESOLUTION NEARLY-ML ESTIMATION OF
SINUSOIDS IN NOISE USING A FAST FREQUENCY
DOMAIN APPROACH
Dr Malcolm D. Macleod
Cambridge University Engineering Department, Trumpington Street,
CAMBRIDGE, CB2 1PZ, UK Tel: +44 1223 332671; fax: +44 1223 332662
e-mail: mdm@eng.cam.ac.uk
ABSTRACT
Estimating the frequencies, amplitudes and phases of
sinusoids in noise is a problem which arises in many
applications. The aim of the metho ds in this paper is
to achieve computational eciency and near-ML perfor-
mance (i.e. low bias, variance and threshold SNR), in
problems such as vibration or audio analysis where the
number of tones may be large (e.g.
>
20). An approach
has recently been published for resolved tones [4]. This
paper extends that frequency domain approachtothe
high-resolution problem.
1 INTRODUCTION
The estimation of the frequencies, amplitudes and
phases of sinusoids in noise is important in many ap-
plications, including radar, sonar, instrumentation, and
audio analysis. In such applications, many of the tones
may not be resolved (i.e. their frequency separations
may be
<
4
=N
rad/sample, where
N
is the block-
length). In cases like these, esp ecially where the number
of tones is large, Maximum Likelihood (ML) estimation
is usually rejected [1] because it requires computation-
ally expensive non-linear optimisation. A recent algo-
rithm [3,4] uses frequency-domain interpolators, cou-
pled with a simple non-linear optimisation strategy,
to obtain nearly-ML estimates in the case of resolved
multiple tones (frequencies separated by at least 4
=N
rad/sample). This pap er extends these results to the
high-resolution case, giving very nearly ML estimates
with much reduced computation.
1.1 Problem denition
The observed discrete signal
y
n
is modelled as
y
n
=
x
n
+
z
n
, where
x
n
is the sum of
M
cisoids
(complex sinu-
soids) and
z
n
is zero-mean complex noise [1] of variance

2
, with independent real and imaginary parts, eachof
variance

2
=
2. If the cisoids have amplitudes
a
i
, phases

i
, and frequencies
!
i
radians per sample,
x
n
can be
written as
x
n
=
M
X
i
=1
b
i
exp(
j!
i
n
)
;
(1)
where
b
i
=
a
i
exp(

i
) is the
complex amplitude
of the
i
th
cisoid. Let the
N
-sample data blocks be written
as column vectors,
y
=[
y
0
;y
1
; :::; y
N

1
]
T
, etc., and de-
ne parameter vectors
b
= [
b
1
;b
2
; :::; b
M
]
T
and
!
=
[
!
1
;!
2
; :::; !
M
]
T
. The problem, assuming
M
is known,
is to estimate
b
and
!
, given
y
=
x
+
z
. (Pure real sig-
nals, and the problem of estimating
M
, are considered
later).
Let matrix
G
have columns which are the cisoid basis
functions at frequencies
!
1
:::!
M
:
G
(
!
) = [
e
(
!
1
)
;
e
(
!
2
)
; :::;
e
(
!
M
)], where
e
(
!
) =
[1
;
exp(
j!
)
; :::;
exp(
j
(
N

1)
!
)]
T
. Then the signal model
(1) can be written
x
=
G
(
!
)
b
. When the noise
z
is
white (i.i.d.) Gaussian, the Maximum Likelihood (ML)
estimate of the parameters (
^
b
;
^
!
) is the one [1] which
minimises
S
=
jj
y

G
(^
!
)
^
b
jj
2
, the sum of squared errors
(SSE) between the estimated signal ^
x
=
G
(^
!
)
^
b
and the
observed signal
y
. For any given estimate ^
!
, the ML
estimate of
^
b
is given by [1]
^
b
(^
!
)=(
G
(
^
!
)
H
G
(
^
!
))

1
G
(^
!
)
H
y
(2)
and the joint ML estimate of
b
and
!
is found by max-
imising

(^
!
)=
y
H
G
(
^
!
)(
G
(^
!
)
H
G
(^
!
))

1
G
(^
!
)
H
y
(3)
by searching over the
M
-dimensional ^
!
. For the single-
tone case (
M
=1),
G
(^
!
)=
e
(^
!
), and the ML estimate
of the scalar ^
!
is obtained [1] by maximising the
peri-
odogram
of the signal
y
,
P
(^
!
) =
y
H
e
(^
!
)(
e
(^
!
)
H
e
(^
!
))

1
e
(^
!
)
H
y
=(1
=N
)
jj
e
(^
!
)
H
y
jj
2
;
since
e
(^
!
)
H
e
(^
!
)=
N
. From (2), the ML estimate of b
is
^
b
(^
!
) = (1
=N
)
Y
o
(
!
), where
Y
o
(
!
) =
e
(^
!
)
H
y
is the
DTFT of
y
. The DTFT of a single cisoid at frequency
!
A
is
e
(
!
)
H
e
(
!
A
)=
D
N
(
!

!
A
), where
D
N
(
!
) =
N

1
X
k
=0
exp(

jk!
)
=exp(

j!
((
N

1)
=
2))
sin(
!N=
2)
sin(
!=
2)
(4)

is a form of the Dirichlet kernel. It has the properties:
D
N
(0) =
N
;
D
N
(
k!
0
)=0if
k
6
= 0; and
D
N
(
!
)

N
,
if
!

!
0
. A traditional way to estimate ^
!
is perform
a coarse search for the periodogram peak, using a zero-
padded DFT, and then rene the estimate by optimisa-
tion. A more ecient approach [3,4] is to lo cate the peak
in the standard DFT and estimate ^
!
using a closed-form
interpolator in the discrete frequency domain.
For
M >
1 the non-linear searchover ^
!
is in general
computationally intensive. The elements of the matrix
T
=
G
(
!
)
H
G
(
!
), whose inverse appears in (3), are
T
mn
=
e
(
!
m
)
H
e
(
!
n
)=
D
N
(
!
m

!
n
)
:
(5)
From (4), the diagonal elements
T
mm
=
N
, and
T
mn
=
T

nm
,so
T
is Hermitian.
1.2 Low resolution multiple tone ML analysis
If
!
m

!
n

!
0
(where
!
0
=2
=N
), the o-diagonal
elements
T
mn
are much smaller than the diagonal el-
ements, and the
m
th
and
n
th
tones produce resolved
peaks in the periodogram. Simple application of a
single-tone estimator to each peak gives biased estimates
[2], caused by the non-zero o-diagonal elements of
T
.
Provided the tone frequencies are separated by at least
2
!
0
(2 `bins'), the bias may be removed [3,4] by a compu-
tationally simple iterative optimisation procedure which
converges rapidly.
2 HIGH RESOLUTION ANALYSIS
The key to the new high resolution approach is to recog-
nise that in typical multi-tone high-resolution problems,
some
of the tones will be resolved, while others will be in
`clusters' with frequency separations
<
4
=N
.Assume
that the frequencies are indexed so that
!
1
< !
2
<
:::<!
M
.Dene a `cluster' of
L
tones, with frequencies
!
j
::: !
j
+
L

1
,by the property that the frequency sepa-
ration between any tone in the cluster and any tone not
in the cluster is much greater than
!
0
. That is, for any
!
m
; j

m

j
+
L

1, and
!
n
;n<j
or
n>j
+
L

1,
wehave
j
!
m

!
n
j
!
0
, hence
j
D
N
(
!
m

!
n
)
j
N
.
Assume that in a given case there are
K
clusters. If
matrix elements with magnitudes

N
are regarded
as negligible, the matrix
T
has approximately the fol-
lowing structure (illustrated for the example of
K
=3
'clusters'):
T
=
2
4
A
1
0 0
0 A
2
0
0 0 A
3
3
5
(6)
in which the square sub-matrices
A
1
-
A
3
correspond
to the clusters, and have non-negligible o-diagonal el-
ements. The overall maximisation in (3) can then be
achieved by independently maximising, for each cluster
k
=1
; :::; K
, the function

k
(^
!
k
)=
y
H
A
k
(
!
k
)(
A
k
(
!
k
)
H
A
k
(
!
k
))

1
A
k
(
!
k
)
H
y
;
(7)
where
!
k
=[
!
j
; :::; !
j
+
L

1
]
T
contains the frequencies of
the L tones in cluster k. Typically many of the `clusters'
will be single isolated tones, so the maximisation (7)
associated with the corresponding submatrix of size 1x1
will be achieved by fast single tone estimation [4].
This reduces the number of
parameters
in each min-
imisation, but computation of
A
k
(
!
k
)
H
y
in (7) still re-
quires
LN
multiplications and additions. A substantial
further improvement can be obtained by extending the
frequency domain approach proposed in [3,4].
2.1 Frequency domain computation
Since the DFT is a linear transform, the ML estimation
task can be formulated equivalently in the discrete fre-
quency domain. Sp ecically,

(^
!
) in (3) can be shown
to be equal to

(^
!
)=(1
=N
3
)
Y
H

(
^
!
)(

(^
!
)
H

(^
!
))

1

(^
!
)
H
Y
(8)
where
Y
is the DFT of
y
and

(^
!
) is the column-by-
column DFT of
G
(^
!
). Similarly,

k
(^
!
k
) in (7) has a
frequency domain equivalent of the form of (8). The
i
th
column of

(^
!
) in (8), being the DFT,
E
(
!
i
), of the
cisoid
e
(
!
i
), is of the form
D
N
(
k!
0

!
i
). The ma jor-
ity of the \energy" (sum squared modulus) of
E
(
!
i
)is
contained in only a few samples centred around the fre-
quency
!
i
. We showed in [3,4] that for single tones, the
use of only 5 DFT samples gives estimates very close to
the true ML estimates; this reduces computation in the
ratio 5
=N
, whichisvery signicant for large
N
. The size
of window required for multi-tone clusters is discussed
below.
Computation of
E
(
!
i
) is /em not carried out by com-
puting the DFT of
e
(
!
i
), but by the much more ecient
direct evaluation of
D
N
(
k!
0

!
i
) using (4). Other ad-
vantages of the frequency domain approach [4] are that
it remains near-optimal in non-Gaussian input noise
z
,
and/or coloured noise, for typical large values of
N
.
Pure real (as opposed to complex) signals are handled by
a simple extension of the above procedure [4]. Only the
parameters of positive frequency tones are estimated,
and corresponding negative frequencies are inferred.
2.2 Size of frequency domain window
For multi-tone clusters, the number of terms of
E
(
!
i
)
H
Y
needed to achieve accurate estimates can be
determined by extending the Cramer-Rao bound (CRB)
calculation approach outlined in [4]. For example, con-
sider the case of two tones. The solid line in Figure 1
shows the CRB for frequency estimation of one of the
tones, normalised to the single-tone CRB for that tone,
and plotted against the frequency dierence between the
two tones, for the worst case relative phase between the
two tones (as shown in [2]).

10−2 10−1 100101
100
101
102
103
RMS ERROR RATIO
FREQUENCY SEPARATION (DFS BINS)
Fig 1. Two tone case. Bound on frequency estimate
rms error (i.e.
p
CRB) normalised to single-tone CRB,
plotted against frequency separation in `bins'. Solid
line: CRB of full ML estimator; Dashed line: CRB of
frequency domain estimator using only 9 DFS samples.
The dashed line in Fig. 1 shows the CRB of the
frequency domain estimator using only 9 terms of
E
(
!
i
)
H
Y
, centred on the frequency of one of the two
tones. The estimator variance is increased by only 1.3
dB at a frequency separation of 0.0625 bins, falling to
less than 0.75 dB for frequency separations of 2 bins or
more. If for example 11 terms are used, these impair-
ments are reduced to 1.06 dB and 0.63 dB respectively.
2.3 Full algorithm
The full algorithm is :

Compute the DFT
Y
.

Repeatedly detect the largest local peak with am-
plitudes above a detection threshold, and apply
single-tone estimation to the new peak (as in [4]).

Apply a single tone bias estimation heuristic [4]
and, for close tones, re-estimate the frequencies by
iteration [4].

Test the residual error over the 5 samples centred
on each peak. If this is suciently small for all
peaks, nish.

For all p eaks with large residual errors, increase the
cluster size L by 1 and re-estimate the L frequencies
and amplitudes of the cluster.

If there are other tones or clusters close enough to
be aected, re-estimate their frequencies.

Test the residual errors for each cluster. If they are
now all small, nish; otherwise continue to increase
the cluster size L (up to a suitable limit) for clusters
with large residual errors, and repeat.
Model order estimation is an intrinsic part of this algo-
rithm. The initial estimate of model order (number of
tones) is simply the number of detected peaks. This is
then increased whenever a cluster size is increased.
3 CONTINUOUS ESTIMATION
The approach described in this paper is being used for
musical audio analysis, where typical blocklengths are
N
=2048 with
M
= 20-50 tones. In applications suchas
this a further requirementistocombine estimates from
sequential (perhaps overlapping) blocks optimally. This
requires knowledge of the estimate variance which, for
a nearly-ML estimator, is approximately equal to the
CRB. However, the CRBs depend strongly on the rel-
ative phase of the tones, and only the CRBs for worst
case phase were published in [2]. A closed-form expres-
sion for the CRBs is desirable. We will consider the two
tone case because it is the most commonly occurring,
and in any case estimator variance increases rapidly as
further close tones are added.
The CRB for frequency estimation of tone
i
can be ap-
proximated by three asymptotes. The rst is the single-
tone CRB,
var

!
i
!
0


6

2
4

2
Na
2
i
:
(9)
This is an absolute lower bound. The second is
var

!
i
!
0


6

2
4

2
Na
2
i
2

(
F
)
4
(10)
where
F
is the frequency separation of the tones in
bins:
F
= (
!
i

!
j
)
=
(2

). This asymptote meets
the single tone bound at
F

1
:
6bins. The third
asymptote depends on relative phase. Dene  =

i


j
+
F
(
N

1)
=N
; this equals the phase dierence
at the block centre (half waybetween sample
N=
2

1
and sample
N=
2). The third asymptote is
var

!
i
!
0


6

2
4

2
Na
2
i
0
:
5

sin
2
()(
F
)
2
:
(11)
Note that this becomes innite as 
!
0 or

.
The complete estimate for the CRB is as follows; it is
max[ (9), min[ (10), (11)]]. Hence max[(9), (10)] is the
bound for worst case phase ( = 0 or

), as rst shown
in [2].
To conrm the above model, Fig. 2 shows the actual
CRBs and the above asymptotic t for two tone esti-
mation, for  = 0
;=
16
;=
8
;=
4
;=
2, as functions of
frequency.

10−2 10−1 100101
100
101
102
103
104
RMS ERROR RATIO
FREQUENCY SEPARATION (DFS BINS)
A
B
C
D
E
Fig 2. Two tone case. Bound on frequency estimate
rms error (i.e.
p
CRB) normalised to single-tone CRB,
plotted against frequency separation in `bins'. A:
 =
=
2; B:  =
=
4; C:  =
=
8; D:
 =
=
16; E:  = 0.
This closed-form expression makes it possible to com-
bine the estimates from successive blocks with the ap-
propriate weighting to reect the (potentially very dif-
ferent) variances of the estimates from the dierent
blocks.
4 CONCLUSIONS
The frequency domain approach described in section 2
achieves high resolution estimation of sinusoids in white
or coloured noise, with performance very close to ML.
It is computationally ecient, particularly for problems
such as audio analysis where there maybemany tones,
many of them resolved.
The CRB model described in section 3 illustrates the
nature of the two-tone CRB more fully than in [2], and
permits fast approximate calculation of the two-tone
CRB. This is of value in continuous estimation of fre-
quencies from successive blocks.
REFERENCES
1. Kay, S.M.,
Modern Spectral Estimation
, Prentice-
Hall, Englewood Clis, NJ, 1988.
2. Rife, D.C., Boorstyn, R.R., "Multiple Tone Pa-
rameter Estimation from Discrete-Time Observations",
BSTJ
,Vol 55, No 9, Nov 1976, pp.1389 - 1410.
3. Macleo d, M. D., "Fast high accuracy estimation of
multiple cisoids in noise",
Signal Processing V
, (Pro c.
Eusipco 90), Elsevier, 1990, pp. 333-336.
4. Macleod, M. D., "Fast Nearly-ML Estimation of the
Parameters of Real or Complex Single Tones or Resolved
Multiple Tones",
IEEE Trans SP
, 1998, Vol.46, No. 1,
pp.141-148 (Jan 98).