Identification of connectivity in neural networks

Abstract

Analytical and experimental methods are provided for estimating synaptic connectivities from simultaneous recordings of multiple neurons. The results are based on detailed, yet flexible neuron models in which spike trains are modeled as general doubly stochastic point processes. The expressions derived can be used with nonstationary or stationary records, and can be readily extended from pairwise to multineuron estimates. Furthermore, we show analytically how the estimates are improved as more neurons are sampled, and derive the appropriate normalizations to eliminate stimulus-related correlations. Finally, we illustrate the use and interpretation of the analytical expressions on simulated spike trains and neural networks, and give explicit confidence measures on the estimates.

Full text

Identification
of
connectivity
in
neural
networks
Xiaowei
Yang*
and
Shihab
A.
Shamma**§
*Systems
Research
Center
and
tElectrical
Engineering
Department
and
the
§University
of
Maryland
Institute
for
Advanced
Computer
Studies,
University
of
Maryland,
College
Park,
Maryland
20742
USA
ABSTRACT
Analytical
and
experimental
pressions
derived
can
be
used
with
stimulus-related
correlations.
Finally,
methods
are
provided
for
estimating
nonstationary
or
stationary
records,
we
illustrate
the
use
and
interpretation
synaptic
connectivities
from
simulta-
and
can
be
readily
extended
from
pair-
of
the
analytical
expressions
on
simu-
neous
recordings
of
multiple
neurons.
wise
to
multineuron
estimates.
Further-
lated
spike
trains
and
neural
networks,
The
results
are
based
on
detailed,
yet
more,
we
show
analytically
how
the
and
give
explicit
confidence
measures
flexible
neuron
models
in
which
spike
estimates
are
improved
as
more
neu-
on
the
estimates.
trains
are
modeled
as
general
doubly
rons
are
sampled,
and
derive
the
stochastic
point
processes.
The
ex-
appropriate
normalizations
to
eliminate
1.
INTRODUCTION
Most
functions
of
the
mammalian
nervous
system
are
performed
by
networks
of
highly
interconnected
neurons.
In
the
experimental
study
of
these
networks,
extracellular
recordings
are
often
employed
to
sample
the
patterns
of
action
potentials
simultaneously
generated
by
several
neurons
(2,
9,
15,
16,
19).
The
correlations
among
the
recorded
firings
of
the
different
cells
are
then
used
as
measures
of
the
type
and
strength
of
their
interconnec-
tions.
Many
such
measures
have
been
proposed
to
accom-
plish
the
latter
task;
they
include
the
cross-interval
histograms,
the
cross-correlation
histograms,
the
cross-
covariance
histogram,
and
the
joint
peri
stimulus
time
(PST)
histogram
(the
scatter
diagram)
(8,
9).
In
all
cases,
the
histograms
provide
statistical
measures
in
support
of
various
hypotheses
such
as
whether
the
two
(or
more)
neurons
under
study
directly
influence
each
other
or
simply
share
common
inputs,
and
whether
the
influences
are
excitatory
or
inhibitory.
There
are
three
basic
difficulties
with
these
methods
that
we
tackle
in
this
report.
The
first
concerns
the
lack
of
flexible
general
analytical
treatments
that
outline
the
relations
between
the
synaptic
connectivities
and
the
correlation
measures
that
are
used
to
estimate
them.
Thus,
while
various
features
in
the
above
mentioned
histograms
may
reflect
qualitatively
the
underlying
con-
nections,
several
parameters
and
conditions
can
render
these
measures
inadequate.
Examples
of
such
difficulties
are
the
differing
integrating
dynamics
of
different
cell
types,
and
the
potentially
severe
errors
due
to
stimulus-
induced
(rather
than
synaptic)
correlations.
Attempts
to
overcome
these
problems,
as
in
the
use
of
the
shuffling
method
to
reduce
stimulus
effects,
are
shown
here
to
be
largely
inadequate.
The
second
basic
shortcoming
of
the
above
correlation
methods
stems
from
the
nonstationarity
of
the
neural
records.
In
constructing
cross-interval
and
cross-correla-
tion
histograms,
counts
are
usually
obtained
not
only
by
averaging
over
different
stimulus
presentation
but
also
by
averaging
over
the
time
duration
of
each
presentation
period.
This
makes
these
two
estimates
inadequate
when
working
with
nonstationary
records
and,
instead,
mea-
sures
based
on
time-dependent
histograms
such
as
the
joint
PST
scatter
diagram
should
be
used
for
the
analysis
(10,
18).
Finally,
it
is
unclear
in
many
existing
methods
how
to
extend
the
analysis
to
more
than
two
neurons,
and
how
to
evaluate
the
degree
to
which
a
pairwise
estimate
is
improved
when
the
records
from
many
other
neurons
are
included.
This
is
a
particularly
important
criterion
as
progress
in
multiunit
recording
technologies
which
prom-
ises
to
increase
significantly
the
number
of
records
of
simultaneously
active
neurons.
To
summarize,
the
objectives
of
this
paper
are
(a)
to
provide
rigorous
analytical
and
experimental
methods
to
estimate
synaptic
connectivities
from
simultaneous
recordings
of
multiple
neurons
that
are
based
on
accurate
and
flexible
neuron
models,
(b)
to
express
synaptic
con-
nectivity
in
terms
of
probability
densities
of
joint
neuronal
firings
and
individual
neuronal
firings
that
can
be
used
with
nonstationary
(or
stationary)
records,
(c)
to
extend
these
methods
from
pairwise
to
multiunit
correlations.
The
paper
is
organized
as
follows.
In
the
next
section,
a
stochastic
nonlinear
neuron
model
is
proposed,
and
the
spike
train
generated
by
the
model
is
expressed
by
a
doubly
stochastic
process.
This
model
will
serve
as
the
fundamental
tool
upon
which
the
analytical
results
are
D;ot
,
I
Dp
.
--
_
- -
---i
o
.c
i
tA
olopnYS.
J.
1iop9ySical
society
Volume
57
May
1990
987-999
0006-3495/90/05/987/
13
$2.00
987

based.
In
section
3,
quantitative
analyses
of
neuronal
connectivities
are
carried
out
through
the
model.
These
include
derivations
of
the
relations
between
the
synaptic
connectivity
and
the
firing
probability
densities,
and
extending
the
pairwise
correlations
to
the
multineuron
case.
In
section
4,
the
results
are
summarized
and
discussed
in
the
context
of
practical
implementations
and
considerations
of
the
accuracy
of
the
estimates.
All
the
analytical
treatments
are
contained
within
sections
2
and
3.
For
the
reader
interested
only
in
using
the
final
expressions,
section
4
outlines
the
results
and
is
sufficient
as
a
guide
for
their
experimental
applications.
Finally,
the
analytical
results
are
simulated
and
discussed
in
section
5.
The
proofs
of
lemmas
and
theorems
are
given
in
the
Appendix.
2.
NEURON
MODEL
{Tk
}k>1
{TTk
k>l
..
I-ISynapse
-
.Nonlinearity
...
1-
h
(t,
s)
r
Tk
}k>1
vA
GTke}k>r
L_S~~~~~~~~9-
erto
The
basic
unit
of
the
nervous
system
which
receives
and
transmits
neural
signals
is
the
neuron.
The
interactions
of
neurons
in
a
network
occur
in
most
cases
through
synaptic
connections
between
them.
Most
synapses
are
found
between
the
axon
terminals
of
a
presynaptic
neuron
and
the
soma
of
dendritic
tree
of
a
postsynaptic
neuron.
Since
there
can
be
many
synapses
between
any
two
neurons,
it
is
impractical
in
modeling
the
neural
network
to
account
for
individual
synapses;
rather,
it
is
more
fruitful
both
for
experimental
investigation
and
mathematical
description
to
consider
the
total
effective
influence
of
one
cell
on
another.
Consider
that
neuron
A
is
influenced
by
a
family
of
neurons
Bi,
i
=
1,
2,
.
..
,
n.
The
model
we
use
is
depicted
in
Fig.
1;
it
is
similar
in
many
respects
to
that
studied
by
Knox
(
11)
and
by
van
den
Boogaard
et
al.
(4).
A
sequence
of
impulses
from
neuron
Bi
is
transformed
into
a
mem-
brane
potential
in
neuron
A.
The
membrane
potential
WA
of
neuron
A
is
represented
by
a
linear
spatial-temporal
superposition
of
all
input
action
potentials
of
neurons
B1,
B2,
.
.
.,
Bn
(including
self
inhibition
and/or
self
excita-
tion),
and
an
unknown
random
potential
U,
which
repre-
sents
the
influence
of
all
other
unobservable
neurons
and
biophysical
factors.
A
sigmoid
function
g
is
used
to
map
the
somatic
potential
as
follows:
W
[
+
E
'
h,(t,
T)
dNB{(T)]
(1)
where
{TBi:
k
=
1,
2,
.
.
.)
are
the
epoch
times
of
spike
train
from
neurons
B,,
and
{NB,(t):
t
-
01
is
the
associated
counting
process,
i.e.,
the
number
of
spikes
arriving
from
neuron
Bi
in
the
interval
(0,
t].
FIGURE
I
A
dynamical
nonlinear
neuron
model,
where
neuron
A
is
considered
as
the
postsynaptic
neuron.
(a)
Neuron
A
is
influenced
by
presynaptic
neurons
B,,
B2,
.
.
.,
B,
(b)
A
synaptic
connection
between
neurons
A
and
B;
the
influences
of
other
neurons
on
neuron
A
are
summarized
by
U,.
(c)
An
equivalent
probabilistic
version
of
the
neuron
model.
The
impact
of
the
random
input
U,
is
now
moved
to
the
spike
generator
where
the
threshold
becomes
random.
A
spike
is
generated
when
the
integrated
membrane
potential,
fO
WA
dT,
exceeds
a
stochastic
threshold
OW(t).
The
membrane
potential
then
discharges
to
a
resting
level
vO,
and
hence
the
input
information
before
the
firing
instant
is
completely
discarded.
Denote
by
hi(t,
s)
the
impulse
response
(not
necessarily
time-invariant)
which
describes
the
total
temporal
influence
of
neuron
Bi
on
neuron
A
from
past
up
to
present,
including
the
conduc-
tion
and
transmission
delay.
A
synaptic
connection
is
said
to
be
excitatory
if
h(t,
s)
2
0
for
all
t,
all
s
in
the
real
line
R;
it
is
said
to
be
inhibitory
if
h(t,
s)
<
0
for
all
t,
all
s
in
R.
For
mathematical
simplicity,
let
us
assume
that
the
nonlinearity
g
has
the
form
of
g(x)
=
aeX,
a
>
0,
i.e.,
that
neuron
A
is
operating
around
threshold
and
is
thus
not
strongly
driven.
This
form
of
nonlinearity
leads
to
a
multiplicative
model,
which
was
used
earlier
by
van
den
Boogaard
(5,
10).
Suppose
further,
without
loss
of
gener-
ality,
that
we
are
interested
in
finding
the
connectivity
between
two
neurons
A
and
B,.
In
the
following
discus-
sion,
we
write
B
=
B,
and
h(t,
s)
=
h,(t,
s)
for
simplicity.
Then,
n
NB,(s)
WA=
g
Ut
+
E E
hi(t,
TkB)
Vt,
a
i
-2
k-IJ
(2)
988
Biophysical
Journal
1990~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
{Tk"
}k>l1
{
Tk
}k>1
988
Biophysical
Journal
Volume
57
May
1990
n
NBAI)
U,
+
T
F
hi(t,TBi),
i-I
k-I

where
V"
is
called
here
the
semi-membrane
potential
due
to
neuron
B
and
is
defined
as
NB(t)
v=
g
h(t,
Tk)].
(3)
k-l
To
account
for
the
firings
of
neuron
A
that
are
due
to
V,
we
can
think
of
the
factor
Zw(t)
=
a/g(U,
+
,
-2
2k
N-i(t)
h,(t,
TB'))
as
a
positive
stochastic
process
with slow
time
variation
relative
to
interspike
intervals.
The
slow
varia-
tion
assumption
is
valid
for
a
neuron
influenced
by
a
large
pool
of
neurons
where
the
contribution
of
each
neuron
is
small.
Therefore,
WA
d
t
[
VA/ZW(T)]d
dT
VA
dT]
IZw(t)
Then
we
define
an
effective
threshold
O(t)
=
ZW(t)0w(t).
A
spike
occurs
whenever
the
threshold
8(t)
is
exceeded
by
the
accumulated
semimembrane
potential,
i.e.,
(4)
VA
dT
>
O(t),
to
two
observable
neurons,
and
show
how
the
connectivity
between
them
can
be
expressed
analytically
in
terms
of
the
neuron
model
outlined
above.
We
then
consider
the
sources
of
uncertainty
in
this
estimate
and
how
they
can
be
reduced
through
added
information
from
neighboring
neurons.
Finally,
we
shall
comment
on
the
critical
nor-
malization
procedures
used
to
remove
the
confounding
effects
of
stimulus
artifacts.
In
the
following
discussion,
we
will
make
use
of
the
PST
histogram
of
a
single
cell
spike
train
which
measures
the
firing
rate
of
a
neuron
with
respect
to
the
stimulus
onset.
Each
bin
of the
PST
histogram
is
an
unbiased
estimator
for
the
probability
density
of
the
average
neu-
ron
firing
over
a
short
period
At
at
instant
t
corresponding
to
that
bin.
Let
us
denote
by
PA(t)
the
conditional
firing
probability
density
of
the
postsynaptic
neuron
given
the
history
of
the
intensity
process
of
the
presynaptic
neuron,
X,
=
v{Vs
s<
tl,
and
the
history
NJA
=
a
NA(s):
s
s
t4
of
spike
train
A,
that
is,
PA,(t)
=
lim
p,(ANA
(t)
IIf
B,
.MA)
At_0
t
(7)
where
to
is
the
instant
of
the
preceding
spike.
In
fact,
Eq.
4
is
equivalent
to
f'O
WA
di
>
0
(t),
described
previously.
Let
Z
and
0O(t),
respectively,
absorb
all
the
randomness
and
all
the
time
variation
of
both
Zw(t)
and
0A(t)
so
that
the threshold
0(t)
is
comprised
of
a
random
variable
Z
and
a
time
function
00(t)
as
0(t)
=
Z0o(t).
(5)
Due
to
the
refractory
period
r
during
which
a
neuron
is
unable
to
produce
a
successive
spike,
the
time
function
can
be
taken
as
simple
as
,
Tk
s
t
<
Tk
+
r
0o(t)
=
[0,
Tk
+
r
-
t
<
Tk+,
(6)
where
00
>
0
is
a
constant,
and
Tk
and
Tk+1
are
the
times
at
which
the
kth
and
(k
+
1)-st
spike
occur,
respectively.
Denoting
by
NA(t)
the
number
of
spikes
in
train
A
during
time
interval
(0,
t],
a
stochastic
counting
process
{NA(t):
t
>
0}
is
associated
with
spike
train
A
with
NA(0)
=
0.
Let
ANA(t)
=
NA(t
+
At)
-
NA(t)
be
the
number
of
spikes
in
an
infinitesimal
duration
At.
We
say
that
a
process
is
orderly
if
P,(ANA(T)
>
1)
=
o(At).
Armed
with
these
general
neuron
models,
we
are
ready
for
the
analysis
of
the
interneuronal
connectivities
deduced
from
the
stochastic
firing
of
several
neurons.
The
firing
probability
density
of
neuron
A
is
defined
as
P,(A,)
=
E[PA(t)]
=
lim
PJ[ANA(t)
=
1I
A-.o
At
(8)
where
the
second
equality
is
obtained
by
interchanging
the
limitation
and
the
expectation
operations.
Because
the
firing
rate
of
a
neuron
is
finite,
this
interchangeability
is
guaranteed
by
the
dominated
convergence
theorem.
This
argument
applies
to
every
similar
situation
throughout
the
paper.
Likewise,
denote
by
PB(S)
iS
the
conditional
firing
probability
density
of
the
presynaptic
neuron
given
the
history
of
the
intensity
process
of
the
presynaptic
neuron
and
the
history
of
spike
train
B,
that
is,
As
PB(S)
=
lim
r
()
Ils
v
0
N0
(9)
We
have
P,(BS)
=
E[P8(s)]
=
lim
Pr(ANB(S)
=
1)
W_0
AS
(10)
Note
that the
individual
PST
histograms
of
neurons
A
and
B
estimate
E[PA(t)]At
and
E[PB(s)]As,
respectively,
and
that
PB(S)
is
not
defined
symmetrically
to
PA(t).
Moreover,
the
joint
PST
histogram
of
the
two
neurons
estimates
E[PAB(t,
S)]tWAs,
where
PAB(t,
s)
represents
the
conditional
joint
probability
density
of
firing
of
neu-
rons
A
and
B,
PAB(O,
S)
P,
(ANA(t)
=
1,ANB(S)
lWfniax(g.S)W,
As
(11)
'1
lim0
(11)
Connectivity
in
Neural
Networks
989
3.
ANALYTICAL
RESULTS
In
this
section,
we
shall
derive
and
elaborate
on
three
basic
results.
We
shall
first
consider
the
simple
case
of
Yang
and
Shamma
Connectivity
in
Neural
Networks
989

and
Pr(At,BS)
=
E[PAB(t,S)I
PJ[ANA(t)
=
I,ANB(S)
=
At,4s-O
AtAs
Recall
that
h(t,s)
represents
the
synaptic
connectivity
between
neurons
A
and
B.
The
three
basic
results
derived
are
as
follows.
Result
1
The
joint
probability
density
of
firing
of
a
presynaptic
and
postsynaptic
neuron
pair
can
be
expressed
as
the
product
of
individual
firing
probability
densities
and
the
pairwise
connectivity,
and
a
corrupting
(uncertainty)
factor
due
to
other
unobservable
influences
on
the
firing
of
A:
P,(Ai,
B.)
=
P,(A,)P,(Bs)y(t,
s)eh(tfS),
where
'y(t,
s)
is
the
corrupting
factor
(y
0)
given
by
E
OA,@)
PB(S)]
y(t,
s)
E[fA(t,
AA)]P(S)]
,Yt
E[fA(t,
OA)I
E[PB(s)1
leads
to
estimators
superior
to
those
produced
by
the
often
employed
shuffle
method
(normalization
by
difference):
Nd
(t,
s)
=
Pr(At,
B,)
-
Pr(A,)
Pr(Bs),
(19)
which
is
the
quantity
that
the
cross-covariance
histogram
estimates.
3.1
Further
relationships
To
discuss
the
derivation
of
the
above
stated
results,
we
will
need
to
utilize
a
few
more
relationships.
Given
a
pair
of
interacting
neurons
(A
and
B),
the
following
lemmas
will
play
an
important
role
in
the
analysis
below.
Let
us
first
define
an
auxillary
function:
fp(t,
')
=
lim
'(ANB(t)
IJOB;
1f
1,
N
W)
a.-o
~At
(13)
(20)
Lemma
1.
PB(t)
can
be
expressed
as
a
map
from
the
semi-membrane
potential
space
of
neuron
B
onto
[0,
oo),
(14)
P8(t)
V,
Eb, [fB(t,
6,)]
(21)
with
where
fA(t,
tO)
=
VA
fe,(a,)
1
-
FOA(a,)
(15)
f
(t,
OB)
fV
(b,)
(22)
where
a,
=
f,'
VA
di,
andf(
*),
Fo(
)
are
the
density
and
the
distribution
functions
of
the
threshold
of
neuron
A,
respectively.
Result
2
The
uncertainty
can
be
reduced
(i.e.,
the
corrupting
factor
can
be
made
closer
to
1)
if
more
interacting
neurons
Cl,
C2,
.
.
.,
Cm
are
observed
simultaneously.
If
Pr(At,Ct)
0
and
Pr(Bs,C,)
0,
then
the
pairwise
connectivity
becomes
h(t,
s)
=
log
p
(A,,
B,
C,)Pp(C,)-loge*
Pr(A
t,
C,)
P,(Bs,
C,)
with
C,
=
{ni,-,
cifires
in
(t,
t
+
At)1,
where
ly*(t,
s)
=
E[fA(t,
--)PI(S)tC,]
E[fA(t,
A)
IC,]
E[PB(s)
I
C,]
(16)
(17)
is
a
quantity
satisfying
ly*
-
I
<
1'
-
II.
If
-y*
is
very
close
to
1,
then
log
y*
can
be
neglected.
Result
3
To
minimize
the
effects
of
the
stimulus
on
the
estimators
of
the
connectivity,
the
normalized
joint
probability
of
firing
given
by
Np(t,
s)
=
P,(At,
BS)
/P,
(A,)
P,(Bs)
(18)
where
b,
V
f'
Vi
dr,
and
f8.
(.),
Fe
(.)
are
the
density
and
the
distribution
functions
of
the
threshold
of
neuron
B,
respectively.
The
expectation
Eb,
[
]
is
taken
with
respect
to
6B.
The
function
PB(
)
can
have
a
very
simple
form.
For
example,
if
the threshold
is
an
exponentially
distributed
independent
random
variable
with
mean
X,
then
PB(t)
=
XVB.
And
in
this
case,
INB(t):
t
2
01
is
a
doubly
stochastic
Poisson
process.
Lemma
2.
The
conditional
expectation
of
the
product
of
the
semi-membrane
potential
of
neuron
A
and
the
firing
rate
of
neuron
B
can
be
expressed
as
dVNA
d
]
=
e
[
V,
(S)
-
E
ds
=]
eh(trs)E
(23)
3.2
Discussion
of
result
1
We
will
first
need
to
derive
an
expression
for
the
firing
rate
of
the
postsynaptic
neuron
(A).
In
general,
the
threshold
0'
of
this
neuron
is
not
an
independent
variable,
because
it
depends
on
all
other
unobservable
inputs
to
the
neuron.
Given
an
arbitrary
value
for
6',
we
define
fA(t
:)
lr
P(ANA(t)
10A;
Wy
NA
)
At-0
~At
(24)
Note
that
fA(t,
6)
is
a
symmetry
to
fB(t,
Or)
defined
in
Eq.
20,
and
is
not
PA(t)
defined
in
Eq.
7.
By
lemma
1
we
990
Biophysical
Journal
Volume
57
May
1990
990
Biophysical
Journal
Volume
57
May
1990

have
Pr(A,)
=
E[fA(t,OA)I
=
E[V
F
I,)
(25)
where
a,
=
f,
VA
dT,
and
fe(*A),
FOA(*)
are
the
density
and
the
distribution
functions
of
the
threshold
of
neuron
A,
respectively.
Similarly,
the
joint
probability
density
of
firing
can
be
expressed
as
Pr(At,
BS)
=
e
h(ts)E[fA(t,
0A)PB(S)].
(26)
Because
the
firing
probability
density
of
the
presynaptic
neuron
is
P,
(BS)
=
E
[PER
(s)
(27)
then
combining
Eqs.
26,
25,
and
27
gives
result
I
with
y
(t,
s)
=
E[fA(t,
)P()
(28)
E[fA
(t,
OA)IE[PB(s)I(8
3.3
Discussion
of
result
2
The
factor
-y(t,
s)
reflects
our
ignorance
of
the
input
to
neuron
B,
or
that
of
the
knowledge
of
the
effective
threshold
OA.
There
are
conceptually
two
ways
in
which
the
uncertainty
can
be
reduced
(i.e.,
'y(t,
s)
1):
(a)
For
a
completely
known
input
{Vs}
(hence
PB(s)
is
determined),
-y(t,
s)
=
1.
This
can
be
achieved
experimen-
tally
if
each
realization
of
spike
train
B
is
identical.
For
instance,
one
may
produce
deterministic
spike
pattern
in
neuron
B
using
electrical
stimulation.
Alternatively,
one
may
construct
a
cross-interval
histogram
(8)
(also
called
a
cross-renewal
histogram
[I])
using
each
spike
in
neu-
ron
B
as
a
reference
time
to
estimate
Pr(A,-,
B,)
(and
hence
h[t
-
s]).
This
is
of
course
only
valid
if
h(t)
is
time
invariant
and
short
relative
to
the
interspike
intervals
of
neuron
B
(i.e.,
the
histogram
trials
are
independent).
One
can
avoid
dependent
histogram
trials
by
constructing
instead
a
conditional
histogram
given
previous
spikes
in
neuron
B
which
estimates
Pr(At,
B,
previous
B
spikes
at
sI,
S2,
.
.
.,
Sk),
where
dependence
is
assumed
to
be
limited
to
at
most
k
consecutive
spikes
in
neuron
B.
The
connec-
tivity
is
then
given
by
h(t
-
s)
=
ln
[P,(At-S,
BSI
previous
B
spikes
at
SI,
S2,
...
,
sk)/P,(A,-SI
previous
B
spikes
at
SI,
S2,
*
Sk
].
(b)
The
alternative
way
is
to
make
fA(t,
A')
more
deterministic
(decreasing
the
variance).
This
occurs
if
we
know
more
about
AA
(comprising
the
intrinsic
threshold
O[t]
and
the
unknown
source
Zw[t]).
Intracellular
mea-
surements
of
neuron
A
obviate
the
need
for
0A(t)
and
give
complete
information
regarding
Zw(t).
However,
where
only
extracellular
recording
are
possible,
information
about
Zw(t)
may
be
obtained
from
measurements
of
more
neurons.
For
instance,
if
the
activities
of
more
interacting
neurons
(C1,C2,
.
.
.,
Cm)
are
available,
we
can
use
a
multiunit
PST
histogram
in
addition
to
the
conventional
joint
and
individual
histograms
to
estimate
P,(At,B
,C,)
P,(C,)
P,(A,,BsIC,)
P,(A,,
C,)P,r(BS,
C,)
P,(A,
P
C,)Pr(BSI
C,)
*(t,s)eh
(29)
where
y*(t,
s)
is
defined
in
Eq.
17.
Because
neurons
(C1,C2,
.
.
.,
Cm)
may
contain
infor-
mation
about
PB(s)
and/or
OA
(for
instance,
if
these
neurons
influence
the
activity
of
either
or
both
neurons
A
and
B),
observing
more
interacting
neurons
makes
fA(t,
0')
less
correlated
with
PB(s).
Consequently,
observ-
ing
more
neurons
makes
-y*
closer
to
one
than
y
is
in
Eq.
28,
and
hence
the
estimator
for
h(t,
s)
is
more
reliable.
The
estimates
computed
from
results
I
and
2
are
superior
to
those
obtained
from
the
cross-correlation
function
which,
as
Knox
pointed
out
in
reference
11,
yields
biased
estimates
of
the
postsynaptic
potential
(PSP)
shape
(i.e.,
the
h[t,
s]),
and
the
amount
of
bias
can
only
be
determined
empirically.
Instead,
in
our
estimates,
the
connectivity
can
be
estimated
by
the
normalized
joint
PST
histogram
along
with
an
uncertainty
factor,
and
this
uncertainty
can
be
explicitly
expressed
under
appropriate
conditions
(see
following
theorem
2).
Furthermore,
it
is
possible
to
reduce
it
by
increasing
the
number
of
observed
neurons.
3.4
Special
case
discussion
To
illustrate
this
with
explicit
analytic
expressions,
three
simplifying
assumptions
will
be
adopted
concerning
the
properties
of
the
postsynaptic
neuron
threshold
OA
(used
in
theorem
1
below)
and
the
distribution
of
the
presynap-
tic
potential
(used
in
theorem
2).
We
start
by
stating
two
of
these
assumptions
and
the
theorems
associated
with
them,
and
then
proceed
to
relate
the
correlation
functions
explicitly
to
the
interneuronal
connectivity
(h[t,
s])
in
a
pair
of
neurons
(A
and
B).
Assumption
1.
The
random
variable
Z
of
the
threshold
in
Eq.
5
is
independent
of
VA,
and
has
an
exponential
pdf:
p2
z
ooe-
(Gz
v0/00
z
<
10,
z
<
vO/00,
(30)
where
00
>
0,
and
vo
is
a
resting
level
of
the
membrane
potential.
This
assumption
is
typically
valid
in
cases
where
neu-
ron
A
is
only
related
to
neuron
B,
i.e.,
it
is
weakly
related
to
any
other
neuron.
It
can
be
verified
that
under
assump-
tion
1,
the
output
spike
train
of
the
neuron
A
is
a
doubly
stochastic
Poisson
process
{NA(t):
t
>
01
with
the
intensity
YagadSam.onciiyi
9
Yang
and
Shamma
Connectivity
in
Neural
Networks
991

process
J{A:
t
>
01,
where
AA
I°
Tk
-
t
<
Tk
+
r
V,
Tk
+
r
:
t
<
Tk+,,
Theorem
2
If
assumptions
1,
2,
and
3
hold,
the
uncertainty
y
in
result
1
can
be
expressed
as
(31)
where
r
is
the
refractory
period.
Note
that
A'depends
on
N',
and
hence
{NA(t):
t
>-
01
is
a
self-exciting
process
with
the
intensity
function
E[AAINA]
(17).
Assumption
2.
The
refractory
period
is
much
smaller
than
any
interspike
interval
and
hence
is
negligible.
Under
this
assumption
AA
does
not
depend
on
NA,
and
hence
the
intensity
process
becomes
the
membrane
poten-
tial
process.
The
following
theorem
is
a
typical
result
of
the
multiplicative
model
first
found
by
Brillinger
(6),
and
we
restate
it
here
in
a
more
general
fashion.
x
y(t
X
s)
=
A
<
A
where
X,
=
J
(e
h(t,s)
_
1)f(r)
dr.
(36)
One
consequence
of
theorem
2
is
that
if
assumption
3
holds
(a
relatively
common
occurrence
[3,
7,
13]),
the
normalized
unconditional
joint
probability
density
Np(t,
s)
can
be
explicitly
evaluated
in
terms
of
these
parameters
as
Theorem
1
(a)
Under
assumptions
I
and
2,
{NA(t):
t
-
01
is
a
doubly
stochastic
Poisson
process
with
the
intensity
process
{VA:
t
-
01.
(b)
In
this
case,
the
conditional
joint
probability
density
of
firing
of
neurons
A
and
B
can
be
expressed
as
PAB(t,
S)
=
PA(t)PB(s)e
(t5)
(32)
for
all
t
and
all
s,
where
h(t,
s)
is
the
interneuronal
connectivity
with
nonzero
transmission
delay.
Theorem
I
states
that
the
conditional
joint
probability
density
of
firing
can
be
expressed
as
the
product
of
the
conditional
individual
firing
densities
and
the
connectiv-
ity.
Thus
the
interneuronal
connectivity
h(t,
s)
could
be
directly
identified
by
h(t,
s)
=
log
PAB(t,
s)
-
log
PA(t)
-
log
PB(s).
(33)
Assertion
b
in
the
above
theorem
includes
the
special
case
where
the
input
process
is
Poisson,
which
was
previously
obtained
by
van
de
Boogaard
et
al.
from
the
expansion
approach
of
the
characteristic
functional
of
the
input
and
the
output
processes
(4).
However,
because
the
membrane
potentials
are
unob-
servable
in
extracellular
recordings,
the
semimembrane
potential
of
the
presynaptic
neuron
B
is
generally
unknown
(hence
PA,
PB,
and
PAB
are
unknown).
There-
fore,
one
must
instead
evaluate
the
normalized
uncondi-
tional
joint
probability
density:
Np(t,
s)
E[PAO
(t,
s)
E[PA(t)I
E[PB(s)]'
(34)
which
is
the
cross-product
ratio
described
in
reference
6.
The
connectivity
h(t,
s)
can
not
be
accurately
estimated
in
general.
Assumption
3.
The
semimembrane
potential
process
of
the
presynaptic
neuron can
be
decomposed
in
the
form
of
V'
=
Xf(t),
where
X
is
gamma
distributed
with
parame-
ters
(X,
v),
and
f(t)
is
a
deterministic
time
function.
Np(t,
s)
A-
eh(,
7
,>
X.
(37)
Therefore,
for
a
given
gamma
distribution
(of
degree
v),
as
the
variance
of
X
(=
v/X2)
becomes
smaller,
X
increases,
and
zy(t,
s)
--
1.
In
other
words,
the
more
is
known
about
V'
(e.g.,
from
recordings
of
additional
neurons),
the
more
accurate
is
the
estimate
of
the
connec-
tivity
between
neurons
A
and
B.
We
will
illustrate
these
results
further
through
simulations
later
in
section
5
(see
Fig.
5).
3.5
Discussion
of
result
3
An
important
factor
in
correctly
interpreting
the
correla-
tions
among
the
activities
of
different
cells
concerns
the
effects
of
the
stimulus.
Specifically,
this
refers
to
the
fact
that
unconnected
cells
may
exhibit
strong
correlations
in
their
firings
purely
due
to
the
fact
that
they
are
driven
by
the
same
stimulus.
To
eliminate
these
effects,
some
form
of
normalization
is
necessary.
In
result
3
we
show
how
the
stimulus
shuffle
alone
fails
to accomplish
this
task.
A
general
discussion
of
different
normalizations
can
be
found
in
reference
14.
If
the
membrane
potential
does
not
vary
much
for
different
stimulus
presentation
(small
variance
of
X),
then
X
>>
7.
Consequently,
from
Eq.
37
we
have
(38)
This
confirms
the
conclusions
established
in
result
2
earlier.
In
contrast
to
the
normalization
used
in
Eq.
18,
the
conventional
cross-covariance
histogram
(which
is
the
modified
joint
PST
diagram
using
the
shuffling
method)
uses
a
difference
normalization
which
estimates
(8,
10):
Nd(t,
s)
=
E[PAB(t,
s)]
-
E[PA(t)]E[PB(s)].
(39)
992
Biophysical
Journal
(35)
h(t,s)
Np(t,
s)
f--
e
Biophysical
Journal
Volume
57
May
1990
992

In
general,
this
expression
is
very
complicated.
However,
if
we
make
use
of
assumptions
3,
it
reduces
to
/eh(fts)
Nd(t,
s)
=
avf(s)M(O')(
-(4X)
where
m(.)
is
the
moment
generating
function
of
X.
If
the
membrane
potential
is
not
varying
too
much
for
different
stimulus
presentation
(X
>>t,),
then
Nd(t,s)
can
be
approximately
written
as
Nd(t,s)
f(s)
(eh('
)-1).
(41)
This
expression
suggests
that
identifying
the
connectivity
here
is
considerably
more
difficult
than
that
of
the
normalization
Np(t,
s)
used
earlier
(see
Eq.
38),
since
quantities
a,
v,
X,
and
function
f(s)
are
generally
unknown.
Nevertheless,
Eq.
41
suggests
that
the
shuffling
method
remains
effective
in
indicating
the
absence
of
a
direct
connection
(i.e.,
when
h
[t,
s]
is
very
small).
because
in
that
case
Nd(t,
s)
is
approximately
zero
regardless
of
the
confounding
terms
(a,
v,
X,
and
function
f[s]).
We
will
illustrate
this
in
simulations
in
section
5
(see
Fig.
6).
hence
it
is
an
N
square
matrix
H.
Element
H
AB
represents
the
average
count
for
coincidence
of
a
spike
in
the
mth
bin
of
train
A
and
a
spike
in
the
nth
bin
of
train
B
over
R
stimulus
presentations,
that
is,
AB
i
H
=-ZA,
B,B
m=
1,2,...,N;
n=
1,2,...,N,
(43)
R
r-
where
Arm
and
Brn
are
the
elements
of
spike
matrices
for
trains
A
and
B,
respectively.
Therefore,
the
matrix
pre-
sentation
of
the
joint
PST
scatter
diagram
is
H=RATB
R
(44)
where
T
denotes
transposition.
The
expanded
joint
PST
histogram
for
multiunit
recordings
(of
M
neurons)
is
then
I
R
H
ilc2...cm
.-
Y-
C,=ni
CC22
...
CTM'
I.2n2nmnRr-l
(45)
where
C'
)
(i
=
1,
2,
.
..
,
M),
is
the
element
of
the
spike
matrix
for
the
ith
neuron.
4.1
Using
the
scatter
plot
to
determine
neuronal
connectivities
4.
EXPERIMENTAL
CONSIDERATIONS
In
the
analysis
of
multineuronal
connectivities,
spike
trains
from
several
neurons
are
recorded
in
response
to
the
repeated
presentation
(e.g.,
R
times)
of
a
stimulus.
Spikes
are
usually
sampled
and
parsed
into
(i.e.,
labeled
by)
small
time
bins
in
which
at
most
one
spike
may
occur
in
each
bin
(which
corresponds
to
the
orderliness
of
the
point
process).
Thus
each
spike
train
is
converted
into
a
discrete
0-1
process,
and
is
further
segmented
into
R
segments,
each
for
one
stimulus
presentation.
Let
A,
be
the
time
bin
corresponding
in
the
nth
bin
associated
with
the
rth
stimulus
presentation.
A
spike
train
can
then
be
represented
by
a
R
x
N
random
matrix
A
with
elements
Arn,
(r
=
1,
2,
.
.
.
,
R;
n
=
1,
2,
.
,N),
which
is
called
here
a
spike
matrix.
The
PST
histogram
reflects
the
stimulus-locked
firing
rate
of
each
single
neuron,
and
it
is
formed
by
taking
average
over
every
column
of
the
spike
matrix,
HA
=RZA,n,
n
=
1,
2,.
.
.,N.
(42)
The
value
of
H
A
counts
the
average
spikes
over
R
stimulus
presentations
in
the
nth
bin
in
a
spike
train
A.
The
joint
PST
scatter
diagram
of
two
neurons
A
and
B
(HzAB,
m
=
1,
2
..
.,N;n=1,2,
N)
measures
the
coincidence
spikes
in
train
A
and
in
train
B
relative
to
stimulus
onset.
It
is
a
two-dimensional
histogram
with
one
axis
(m)
for
train
A
and
the
other
axis
(n)
for
train
B,
and
The
correlations
between
a
pair
of
recorded
neurons
(A
and
B)
can
be
computed
from
the
experimental
estimate
of
the
expression
of
result
1,
i.e.,
h
(t,
s)
=
log
(N
(t,
s)\
-y
(t,
s))
(E[PA(t)
E
[PB(
s)]
where
E[PA(t)]
and
E[PB(s)]
represent
the
PST
histo-
grams
of
firings
of
the
neuron
pair,
E(PAB(t,
S)]
is
their
scatter
plot,
and
y(>O)
is
the
corrupting
factor
repre-
senting
the
uncertainty
in
the
estimate
due
to
the
influences
of
other
unobserved
neurons
and
biophysical
factors.
Thus
in
terms
of
bin
numbers
m
and
n,
the
above
equation
can
be
written
as
HAB
log
HmHn
=
h
(mAt,
nAt)
+
log
[y
(mAt,
nAt)].
(46)
In
the
case
of
time
invariant
connectivities,
h(t,
s)
becomes
h(t
-
s),
and
the
correlation
peak
becomes
a
band
that
runs
parallel
to
the
principal
diagonal
(t
-
s
=
0).'
'Note
that
one can
detect
further
correlations
in
the
unnormalized
scatter
plot,
such
as
the
more
diffuse
bands
of
time-invariant
common
inputs
(20).
Of
course,
these
features
are
intentionally
removed
by
the
normalization
because
they
do
not
reflect
direct
connectivities
within
the
neuron
pair.
Yang
and
Shamma
Connectivity
in
Neural
Networks
993
Yang
and
Shamma
Connectivity
in
Neural
Networks
993

In
the
practical
application
of
Eq.
46,
the
confounding
-y(t,
s)
contributions
are
not
known.
However,
the
analy-
sis
shows
that
additional
simultaneous
recordings
can
be
used
to
reduce
these
uncertainties.
Therefore,
by
using
the
additional
data,
the
improved
estimator
for
h(t,
s)
becomes
HABC.cM
H
C3---CM
log
mnm
...m
m...m
log
HAC3
CM
HBC3
CM
mm...m
nm...m
=
h(mAt,
nAt)
+
log
[y*
(mAt,
nAt)],
(47)
where
C:C2
.C
are
simply
the
joint
multidimensional
scatter
plots
defined
in
Eq.
45,
and
the
uncertainty
factor
'y*
(<y)
is
defined
in
Eq.
17.
The
estimates
of
Eqs.
46
and
47
are
illustrated
in
network
simulations
in
section
5.
Other
methods
to
reduce
y
are
discussed
in
section
3.3.
4.2
Establishing
confidence
measures
on
the
estimates
The
histograms
are
random
variables
subject
to
fluctua-
tions.
Hence,
it
is
important
to
determine
upper
and
lower
bounds
such
that
we
assume
a
connection
between
neu-
rons
A
and
B
whenever
these
bounds
are
surpassed.
By
the
law
of
large
numbers,
H
AB
converges
to
E[PAB(t,
S)],
so
does
HA
to
E[PA(t)]
and
HB
to
E[PB(s)]
almost
surely
as
R
oc.
Therefore,
if
neurons
A
and
B
are
independent,
by
theorems
on
limiting
distributions,
HAB
mn
HAH
as
R
(48)
almost
surely.
The
hypothesis
71o
is
that
the
two
neurons
are
statistically
independent,
which
is
supported
by
E[PAB(t,
s)]
=
E
[PA(t)]
E
[PB(s)].
(49)
And
the
alternative
hypothesis
X,
is
that
the
two
neurons
depend,
which
is
described
by
E[PAB(t,
S)]
EE[PA(t)]
E[PB(s)].
(50)
One
expects
HA/BI(HgAHB)
to
be
close
to
1
if
hypothesis
710
is
true.
Conversely,
if
the
amount
it
deviates
from
1
exceeds
a
bound
b,
one
accepts
hypothesis
X,.
Now
for
a
given
significance
level
a,
we
need
to
find
the
bound
b
satisfying
PI
(
Hmn_
1
>
b
I
HA,
HB
Yifo
=a.
H
AHB
n
(51)
The
hypothesis
testing
is
stated
as
the
following
theorem.
Theorem
3
Let
b be
a
bound
which
divides
a
critical
region
for
the
hypothesis
testing.
One
announces
that
there
is
a
depen-
-64
IC-
654
-64
k-0
a
b
Biophysical
Journal
Volume
57
May
1990
FIGURE
2
Simulations
for
pairwise
excitatory
and
inhibitory
correlations.
(a)
Excitatory
coupling
h(t,
s)
=
0.8e-20(Q-),
t
>
s.
Shown
is
the
two-dimensional
normalized
scatter
plot
generated
by
the
spike
trains
of
the
two
neurons;
below
it
is
the
histogram
Gk
that
results
from
collapsing
the
scatter
plot
along
the
principal
diagonal.
It
corresponds
to
the
function
Np(k)
=
eh(k).
The
upper
and
lower
bound
lines
represent
the
95%
confidence
measure.
(b)
Inhibitory
coupling,
similar
to
a
for
h(t,
s)
=
-
3.0e-2(ts),
t
>
s.
I
-
Volume
57
May
1990
994
Biophysical
Journal

dence
between
the
two
observed
neurons
if
|Hnn-
_
1
>
b.
HAH
estimator
for
the
time-invariant
connectivity
h(t,
s)
=
h(t
-
s).
This
enables
us
to
establish
a
bound
such
that
(52)
For
the
given
significance
level
a
of
false
announcement
of
dependence,
the
bound
can
be
approximately
calcu-
lated
by
b
/b
1
-HAHB
RHAHB
(53)
Pr(IGk
-
II
>
bk
I
=)
a.
(59)
Theorem
4
Given
a
significance
level
a,
let
bk
be
a
bound
of
critical
region
satisfying
the
above
equation,
then
bk
may
be
approximately
written
as
V2;min(N,N-k)
2
bk
n-max(I,I-k)
an+k,n
N-IkI
Eb,
(60)
where
the
value
of
Eb
is
determined
from
4((b)
=1
-
2'
(54)
and
4(x)
=
I
/
V
f
x/2
dx.
The
function
F(x)
is
usually
available
as
the
standard
normal
distribution
table.
For
example,
a
=
0.05
gives
Eb
=
1.96.
The
above
theorem
implies
that
element
Hm/
(HAHB)
of
the
normalized
joint
PST
diagram
has
a
conditional
expectation
value
I
and
an
approximate
con-
ditional
variance
I
-
HAHB
mr
n
(55)
given
the
values
of
HA
and
H'
under
hypothesis
W0.
Because
HA
and
HBare
usually
very
small
and
R
is
fairly
large,
this
approximation
is
close
to
a
recent
result
by
Palm
et
al.
(14)
where
their
conditional
variance
is
2
(I
HA)(
-
Hn)
crmn
-(R
-
1)HAHm
(56)
under
hypothesis
9O.
The
bound
dividing
the
hypothesis
regions
can
be
made
more
useful
in
neural
networks
with
time-invariant
con-
nectivities.
Let
wm
reflect
the
fluctuation
in
the
normal-
ized
joint
PST
diagrams
such
that
HA
=
-y(mAt,
nAt)eh((m
n)At)
+
Wmi,
Hm
Hn
(57
and
the
mean
of
wn
is
zero.
Let
k
=
m
-
n.
A
collapsed
version
can
be
generated
by
averaging
over
diagonals
of
the
normalized
joint
PST
diagram.
This
collapsed
version
is
a
one-dimensional
histogram
Gk
expressed
by:
min(N,N-k)
AB
N
|ki
n-max(ljl-k)
Hn+kHn
k
=-N
+
1,
..
.
-1,O,
1,.
..
N-I
,
(58)
where
k
=
0
is
the
collapsed
point
of
the
principal
diagonal.
Because
averaging
reduces
the
fluctuations
(the
average
of
wm
has
a
smaller
variance),
Gk
is
a
better
where
Eb
is
the
same
as
in
theorem
3,
and
amn
is
given
by
Eq.
55.
Furthermore,
bk
will
reduce
to
(61)
b
bk
L
when
a'
I's
are
taken
as
constants.
I
FIGURE
3
Interaction
among
three
neurons.
The
network
structure
is
displayed
on
the
top
graph:
neuron
B
inhibits
neuron
A
and
excites
neuron
C,
and
neuron
C
excites
neuron
A.
hAg(t)
=-
1.8e
20:s
hAC(t)
=
3.6e-
20,
and
hcB(t)
=
2.Oe
-20.
The
top
curve
gives
the
theoretical
connectivity
from
formula
29
with
y*(t,
s).
The
middle
one
is
the
correlation
curve
corresponding
to
formula
46
generated
from
spike
trains
A
and
B
only.
The
correlation
is
so
distorted
that
actual
inhibition
becomes
a
false
excitation
(which
is
actually
due
to
a
strong
excitatory
input
from
neuron
C).
The
bottom
curve
shows
the
tripartite
correlation
according
to
formula
47,
which
displays
the
correction
inhibitory
sign
for
the
connectivity.
Connectivity
in
Neural
Networks
995
;0:
iiE
S,~~~~~~~~~~~~.
+~~~~~~~~~~~~~~~.
..
....R-.
i:..-......
...
.
.:::
:
-..L.-
:
..
Connectivity
in
Neural
Networks
995
Yang
and
Shamma

This
theorem
indicates
that
the
critical
region
is
enlarged
(the
bound
value
decreases)
when
the
collapsed
version
of
the
normalized
joint
PST
histogram
is
used.
5.
SIMULATIONS
AND
DISCUSSION
To
illustrate
the
nature
of
the
estimates,
uncertainties,
and
bounds
derived
earlier,
we
show
the
results
from
simulations
of
networks
of
excitatory
and
inhibitory
neu-
rons.
The
neuron
model
used
for
the
simulations
is
depicted
in
Fig.
I
c
where
the
nonlinearity
g(x)
=
ex
and
the
random
threshold
has
an
exponential
distribution
with
mean
1.
The
resulting
output
process
of
each
neuron
is
a
doubly
stochastic
Poisson
process
with
the
intensity
pro-
cess
g[1i
zk
i',)
h,(t,
T)b],
where
the
input
process
IT4:
k
=
1,
2,
.
.
.1
is
the
output
process
of
neuron
Bi.
In
the
first
case
(Fig.
2),
for
pairwise
excitatory
and
inhibitory,
time-invariant
connections
are
estimated
using
the
normalized
scatter
plots;
the
uncertainty
factor
(-y)
is
equal
to
1.
The
upper
plots
show
the
two-
dimensional
normalized
scatter
plots.
The
correlations
appear
as
bands
along
the
principal
diagonal
because
h(t,
s)
is
time-invariant.
Hence,
the
scatter
plot
can
be
collapsed
along
this
axis
to
produce
the
lower
histograms.
Note
that
time
variations
in
h(t,
s)
(e.g.,
due
to
poststimu-
lus
adaptation)
do
not
allow
this
reduction.
Consequently,
it
should
only
be
performed
on
the
portions
of
the
neural
record
that
display
obvious
stationary
behavior.
In
both
simulations
of
Fig.
2,
the
predicted
analytical
estimates
are
also
plotted
for
comparison,
together
with
the
bound
lines
for
the
confidence
measures
(determined
by
theorem
4).
To
illustrate
the
effects
of
the
uncertainty
factor
y,
we
examine
in
Fig.
3
the
interactions
among
three
neurons
with
time-invariant
connectivities.
Here,
neuron
A
is
inhibited
by
neuron
B
and
excited
by
neuron
C,
and
neuron
C
is
in
turn
excited
by
neuron
B.
Because
of
the
interactions
between
B
and
C,
the
threshold
in
neuron
A
is
no
longer
independent
of
the
firings
of
B.
Thus,
if
we
attempt
to
identify
the
connectivity
between
neurons
A
and
B
from
pairwise
recordings,
the
estimates
will
be
-64
k=O
64
a
-64
k=O
64
b
FIGURE
4
Comparison
of
the
preferred
normalization
with
the
difference
normalization
(shuffle
method).
(a)
The
absence
of
a
direct
connection
case
(h
=
0).
Neurons
A
and
B
have
a
common
input
source:
a
neuron
driven
by
a
stimulus.
The
connection
strength
from
the
common
input
is
w
=
1.
Top
curve
gives
the
collapsed
version
of
the
joint
PST
histogram
without
any
normalization.
The
correlation
peak
is
purely
due
to
stimulus
effects.
Middle
curve
represents
the
difference
normalized
correlation.
Bottom
curve
shows
the
preferred
normalized
correlation
curve.
Both
methods
perform
well
in
indicating
the
absence
of
connection
between
A
and
B.
(b)
The
presence
of
a
direct
connection
case
(h
.
0):
Neurons
A
and
B
have
a
common
input
source
as
in a,
and
in
addition,
a
direct
synaptic
connectivity
from
B
to
A,
hAB(t)
=
0.4e-2'.
Top
curve
gives
the
theoretical
correlation
predicted
from
Np(k)
=
eh(k).
Middle
curve
shows
the
difference
normalized
correlation.
Although
the
connectivity
is
weak
(only
0.4),
the
large
sharp
peak
in
the
correlation
leads
to
a
false
impression
of
high
excitatory
connectivity,
which
is
in
fact
due
to
stimulus
effects.
The
bottom
curve
shows
the
preferred
normalized
correlation,
which
is
very
close
to
the
theoretical
function
0.4e-20'.
996
Biophysical
Journal
Volume
57
May
......
1-
-
d-
....I.......
1.
....
Id-
111.1
....
III-
-
I,
.,
-
d-
.-
I,
I-
1.
L
I.,,,,
I.......
L-
1.
II,,.,,
...
...I...
-
'I
.......I.......
I......
.j
.......I.......I.......I.......
I.......I....
1.
......
1..
.....I......
996
Biophysical
Journal
Volume
57
May
1990

contaminated
by
the
uncertainty
factor.
The
top
curve
in
Fig.
3
first
shows
the
"target"
theoretical
connectivity
obtained
from
the
multirecording
estimate
given
by
for-
mula
29
with
-y*(t,
s)
=
I
(i.e.,
ehA(I-S)).
If
neuron
C
is
ignored,
the
pairwise
estimate
of
ehAl(-S)
is
shown
as
the
middle
curve
in
Fig.
3
(corresponding
to
formula
46).
The
correlation
is
so
distorted
that
actual
inhibition
becomes
false
excitation
because
of
the
strong
excitatory
activity
from
neuron
C.
To
correct
the
erroneous
correlation,
we
have
to
use
the
information
from
the
third
neuron.
The
tripartite
correlation
according
to
formula
47
is
displayed
at
bottom
of
Fig.
3,
which
is
much
closer
to
the
analytical
estimate.
Fig.
4
compares
the
preferred
normalization
with
the
difference
normalization
(shuffle
method)
under
two
situations.
In
the
absence
of
a
direct
connection,
the
shuffle
method
provides
accurate
indication
of
the
lack
of
synaptic
inputs
between
the
two
neurons.
However,
in
the
presence
of
a
direct
connection,
the
shuffle
method
fails
to
remove
completely
the
stimulus
correlations
as
indicated
by
the
deviation
from
the
analytical
results.
Instead,
the
normalization
suggested
in
this
paper
performs
well
in
both
cases.
In
conclusion,
the
above
simulations
confirm
the
pro-
posed
theory.
The
neuron
model
adopted
is
quite
general
because
(a)
the
synaptic
connectivity
h(t,
s)
represents
a
time-varying
system,
(b)
the
processes
representing
spike
trains
are
not
necessarily
Poisson
processes,
and
(c)
the
nonlinear
function
g(x)
=
aex
is
an
approximation
of
aexl
1
+
aex
when
aex
<<
1,
meaning
that
the
neuron
is
operating
at
low
firing
rates.
Moreover,
our
analytical
results
I
and
2
do
not
dependent
on
any
further
assump-
tions.
Although
the
three
simplifying
assumptions
were
made
in
order
to
see
result
3
more
clearly,
we
did
not
use
assumption
3
in
the
simulations
of
Fig.
4.
The
analysis
presented
in
this
paper
also
points
to
the
following
sobering
conclusion:
For
multiunit
correlation
analysis
to
play
a
useful
role
in
establishing
the
basic
circuitry
of
the
nervous
system,
new
technologies
have
to
be
developed
for
stable,
multiunit
recordings.
These
requirements
stem
from
the
need
for
extended
simulta-
neous
recordings
from
many
cells
to
construct
adequate
scatter
histograms
and
to
minimize
inherent
uncertainty
due
to
unobserved
but
related
activities.
Unfortunately,
neither
of
these
requirements
are
easily
met
at
present,
although
extensive
efforts
towards
this
goal
are
underway
through
the
use
of
silicon-based
microelectrode
arrays
(12).
APPENDIX
Proof
of
lemma
1.
The
threshold
of
neuron
B,
which
is
a
continuous
random
variable,
has
the
probability
density
function
and
the
distribu-
tion
function
denotedf9o(x)
and
FOB(x),
respectively.
Let
b,
=
fo
VB
dT,
where
to
is
the
occurrence
instant
of
the
previous
spike.
From
definition
of
PB(t)
we
have
PB(t)
=
Eb,[fB(t,
Or)],
where
the
expectation
Eb,
[.]
is
taken
with
respect
to
OfB.
And
f8(t,
)
=
lim
P,(bt+AI
2
O
lb,
<
OB;
yB,
VB)
f
~~~~~~t
P,(b,
<
OB
bt
b+,IIB,
vNB)
At-0
AtP,(bt
t
t')
17.
vt
VB
foa(bt)
'
-1Fos(bt)
(62)
(63)
Furthermore,
if
the
threshold
is
exponentially
distributed
with
mean
A,
then
fep(b,)/[I
-
Fes(b,)]
=
A,
and
hence
P,(t)
=
X
V'.
In
this
case,
PB(t)
does
not
depend
on
1WB(t)1,
and
lNB(t)l,,0
evolves
without
aftereffects.
Proof
of
lemma
2.
Because
AN,(t)
can
take
values
0
and
1
only,
by
Eq.
3
we
have
E
[VtA
As
tVS
JS]
E
[a
exp{
h(t,
TB)}
AN,(S)
=
yiB
VS
NB]
Pr(AN8(S)
=
1
B,
/Vs)
(64)
As
For
I
>
s,
the
conditional
expectation
in
the
above
equation
can
be
written
as
E
[a
exp
t)(t
T)
ANB(s)
=
1;
7B
NB
=
E
[a
exp
h(t,
Tk)}
exp
{h(t,
s
+
As)J
NI)
*
exp|
h
(t,
Tk-
].
k
-N,g(s
+As)
+
I
(65)
which
becomes
(66)
as
As
goes
to
0.
Because
P8(s)
is
a
measurable
function
with
respect
to
a(9
5
x
NWy),
we
obtain
E[VA
dNB(S)]
*t)E[
VA
yI
X,
N
JV
]
PB(S)]
eh(s)
E[V,
PB(s)].
(67)
Fort
s,
we
have
VA
AN"S)
W
s
]
E
[
VA
qy,6
N6
B]
EL
NB(S)
YX
B
V
B]
VE[v
I7f,
.N
V]
Pr(ANB(s)
=
iji,
NB)
(68)
As
Yang
and
Shamma
Connectivity
in
Neural
Networks
997
[
VAI
Yi
B,
N
Bi,
eh(l,s)
E
t t
s
Yang
and
Shamma
Connectivity
in
Neural
Networks
997

hence
E[
VAdNB(s)]
[
ds]
=E[E(
VA,
g
Bs
VNB)
pB(S)]
E
[VtA
PB(S)]
(69)
Because
h(t,
s)
represents
a
synaptic
connectivity,
which
is
a
causal
system
with
nonzero
transmission
delay,
h(t,
s)
=
0
for
t
c
s.
Thus
lemma
2
holds
for
all
t
and
s.
Proof
of
theorem
1.
Suppose
that
assumptions
1
and
2
hold,
and
that
the
threshold
0,
has
an
exponential
distribution
with
mean
1.
Then
p,[mNA(t)
=
IlIwA
-
V]
A
At
_
v
lim
=
,.
at-0
/t
(70)
hence
spike
train
NA(t)
represents
a
doubly
stochastic
Poisson
process
with
the
intensity
process
{VA:
t
2
01.
Therefore,
by
Eqs.
31
we
have
(71)
where
N(0,
1)
is
denoted
as
a
standard
Gaussian
random
variable.
Then
if
spike
trains
A
and
B
are
uncorrelated,
we
approximately
write
RH
AB
-
RE[H
AB]
RHAB
-
RHAHB
RHA(l
-HVV)
RHAHn(l
-HAH)
This
means
that
Eq.
78
can
be
approximately
written
as
Pw(eIN(Or
I
>
eb
|
9o)
=
a,
where
bRH
AHB
b
VRHAHB(1
HmHn
)
which
results
in
an
expression
of
the
bound
as
b
RH
AH
(80)
(81)
(82)
(83)
We
choose
At
and
As
such
that
s
<s
+
As
<
t
<
t
+
At,
or
t
<t
+
At
<
s
<
s
+
As.
Because
Poisson
process
is
an
independent
increments
process,
the
conditional
probability
given
the
firing
histories
of
neurons
A
and
B
can
be
split
into
Pr[ANA(t)
=
1,
/ANB(S)
=
1I
Ift
11gvs]
=
Pr[ANA(t)
=
I1f
A]
Pr[ANB(S)
=
1I?if,
1yVs]A
(72)
By
Eq.
31,
the
first
factor
is
Pr[ANA(t)
=
I
|jA
]
=
V'At
+
o(At).
(73)
We
write
the
second
factor
as
Pr(A\N8(S)
=1
IW,
XhV
)
=
E[ANB(s)
IIi
IV
(74)
and
we
have
Pr[ANA(t)
=
1,
ANNB(S)
=
II1f4,
71jv
]
=
E[VA'ANB(S)L19A,
1B
Vs]
+
O(At).
By
taking
average
over
the
a-field
gy
A,
we
obtain
PAB(t,
s)
=
E
[A
dN(s)
which
is,
by
the
proof
of
lemma
2,
PAB(t,
S)
=
e(1-s)E
[VAI7B
]PB(S)
=
eh(s)PA(t)PB(s).
(75)
(76)
The
value
of
Lb
is
determined
by
¢((b)
=
I
-
a
'
(84)
where
4D
(x)
=
I
x
e_x
/2
dx.
The
above
arguments
imply
that
element
I4i,'/(H.H')
of
the
normal-
ized
joint
PST
diagram
has
a
conditional
expectation
value
1
and
an
approximate
conditional
variance
2
lHrnn
2mn
-RHmH
RHAH
HB
(85)
under
hypothesis
9t0.
Proof
of
theorem
4.
let
us
note
that
under
hypothesis
N0,
HAB/
(H'H9)
is
approximately
Gaussian
distributed
with
mean
1
and
vari-
ance
a
n.
Hence,
I
min(N,N-k)
lGk
-
II
N
-
E
n+knNn(O,
1)
(86)
N-k
n-
max(l,lI-k)
where
each
Nn(O,
1)
approximately
has
a
standard
Gaussian
distribu-
tion
expressed
by
(77)
Proof
of
theorem
2.
This
can
be
found
in
reference
20.
Proof
of
theorem
3.
For
a
given
significance
level
a,
we
need
to
find
a
bound
b
satisfying
(
|H
BA
P.
Hm
-1
I
>bIH
AH
B;
L.I=a.
I
HH
B
m
n
(78)
RAB
RHA
HB
N
(0,
1)
=
RHn+k.n
-
n+kHn
VRHn+kH"(1
-
H
+k
H)
(87)
and
a.,
is
given
in
Eq.
55.
Therefore,
Gk
-
1
is
approximately
Gaussian
distributed
with
zero-mean
and
variance
Let
us
remember
that
RH,,
,is
binomially
distributed
with
parameters
(R,
E[PAB(t,
s)])
and
that
HARB
_
HAHB
almost
surely
under
110.
By
the
central
limiting
theorem
and
theorems
on
limiting
distributions,
RHAB
-
RE
[HgAB]
A/RHAn'(1
_HAB)
N(OI)
as
R-
(79)
1
min(N,N-k)
Var(k-
1)
=
(N
IkI)2
E-
k
n+k,n
(N-
Ik
n-max(l,l
-k)
where
mutual
independence
of
NJ(O,
1)
is
assumed.
Let
bk
=
Var(Gk
-
1)
998
Biophysical
Journal
Volume
57
May
1990~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(88)
(89)
VA
yi
B).
PA(t)
==
E
(
tt
998
Biophysical
Journal
Volume
57
May
1990

we
obtain
Pr(IGk
-
II>
bk
HA,
Hn;
Wo)
=
2[1
-
4(Eb)I-
(90)
If
all
an's
are
the
same,
observing
the
bound
b
in
theorem
3
completes
the
proof.
The
authors
would
like
to
express
their
appreciation
to
Dr.
James
Fleshman
and
to
the
referees
for
the
valuable
comments
and
sugges-
tions.
This
work
was
supported
in
part
by
grants
from
the
Whitaker
Founda-
tion
and
the
Naval
Research
Laboratory.
Received
for
publication
26
May
1989
and
in
Final
Form
12
October
1989.
REFERENCES
1.
Abeles,
M.
1982.
Local
Cortical
Circuits:
An
Electrophysiological
Study.
Springer-Verlag,
Berlin.
2.
Ables,
M.,
and
M.
H.
Goldstein.
1977.
Multispike
train
analysis.
Proc.
IEEEE
(Inst.
Electr.
Electron.
Eng.)
65:762-772.
3.
Bishop,
P.
B.,
W.
R.
Levick,
and
W.
0.
Williams.
1964.
Statistical
analysis
of
the
dark
discharge
of
lateral
geniculate
neurones.
J.
Physiol.
(Lond.).
170:598-612.
4.
van
den
Boogaard,
H.,
G.
Hesselmans,
and
P.
Johannesma.
1986.
System
identification
based
on
point
processes
and
correlation
densities.
1.
The
nonrefractory
neuron
model.
Math.
Biosci.
80:143-171.
5.
van
den
Boogaard,
H.
1988.
System
identification
based
on
point
processes
and
correlation
densities.
11.
The
refractory
neuron
model.
Math.
Biosci.
91:35-65.
6.
Brillinger,
D.
R.
1975.
The
identification
of
point
process
systems.
Ann.
Probability.
3:909-924.
7.
Correia,
M.
J.,
and
J.
P.
Landolt.
1977.
A
point
process
analysis
of
the
spontaneous
activity
of
anterior
semicircular
canal
units
in
the
anesthetized
pigeon.
Biol.
Cybern.
27:199-213.
8.
Gerstein,
G.
L.
1970.
Functional
association
of
neurons:
Detection
and
interpretation.
In
The
Neurosciences
Second
Study
Pro-
gram.
F.
0.
Scmitt,
editor.
Rockefeller
University
Press,
New
York.
648-661.
9.
Gerstein,
G.
L.,
and
D.
H.
Perkel.
1969.
Simultaneously
recorded
trains
of
action
potentials:
analysis
and
functional
interpretation.
Science
(Wash.
DC).
148:828-830.
10.
Habib,
M.
K.,
and
P.
K.
Sen.
1985.
Non-stationary
stochastic
point-process
models
in
neurophysiology
with
applications
to
learning.
In
Biostatistics:
Statistics
in
Biomedical,
Public
Health
and
Environmental
Sciences.
P.
K.
Sen,
editor.
Elsevier/North-
Holland,
Amsterdam.
481-509.
11.
Knox,
C.
K.
1974.
Cross-correlation
functions
for
a
neuronal
model.
Biophys.
J.
14:567-582.
12.
Kruger,
J.
1983.
Simultaneous
individual
recordings
from
many
cerebral
neurons:
techniques
and
results.
Rev.
Physiol.
Biochem.
Exp.
Pharmacol.
98:177-233.
13.
Nakahama,
H.,
H.
Suzuki,
H.
Yamamoto,
S.
Aikawa,
and
S.
Nishioka.
1968.
A
statistical
analysis
of
spontaneous
activity
of
central
single
neurons.
Physiol.
&
Behav.
3:745-752.
14.
Palm,
G.,
A.
M.
H.
J.
Aertsen,
and
G.
L.
Gerstein.
1988.
On
the
significance
of
correlations
among
neuronal
spike
trains.
Biol.
Cybern.
59:1-11.
15.
Perkel,
D.
H.,
G.
L.
Gerstein,
and G.
P.
Moore.
1967.
Neuronal
spike
trains
and
stochastic
point
processes.
1.
The
single
spike
train.
Biophys.
J.
7:391-418.
16.
Perkel,
D.
H.,
G.
L.
Gerstein,
and
G.
P.
Moore.
1967.
Neuronal
spike
trains
and
stochastic
point
processes.
II.
Simultaneous
spike
train.
Biophys.
J.
7:419-440.
17.
Snyder,
D.
L.
1975.
Random
Point
Process.
John
Wiley
&
Sons,
New
York.
18.
van
Stokkum,
l.
H.
M.,
P.
1.
M.
Johannesma,
and
J.
J.
Eggermont.
1986.
Representation
of
time-dependent
correlation
and
recur-
rence
time
functions.
Biol.
Cybern.
55:17-24.
19.
Yang,
X.,
and
S.
A.
Shamma.
1988.
A
totally
automated
system
for
the
detection
and
classification
of
neural
spikes.
IEEE
(Inst.
Electr.
Electron.
Eng.)
Trans.
Biomed.
Eng.
35:806-816.
20.
Yang,
X.
1989.
Detection
and
classification
of
neural
signals
and
identification
of
neural
networks.
Ph.D.
Thesis.
Electrical
Engi-
neering
Department,
University
of
Maryland,
College
Park.
In
press.
Yang
and
Shamma
Connectivity
in
Neural
Networks
999