Non-parametric laser and video data fusion: Application to pedestrian detection in urban environment

Abstract

In urban environments, pedestrian detection is a challenging task for automotive research, where algorithms suffer from a lack of reliability due to many false detections. This paper presents a multisensor fusion method based on a stochastic recursive Bayesian framework also called particle filter which fuses information from laser and video sensors to improve the performance of a pedestrian detection system. The main contributions of this paper are first, the use of a non-parametric data association method in order to better approximate the discrete distribution and second, the modeling of the likelihood function with a mixture of Gaussian and uniform distributions in order to take into account all the available information. Simulation results as well as results of experiments conducted on real data demonstrate the effectiveness of the proposed approach.

Full text

Non-parametricLaser and VideoDataFusion:Applicationto
PedestrianDetectioninUrbanEnvironment
S.Gidel,C.Blanc,T.Chateau,P.Checchinand L.Trassoudaine
LASMEA-UMR6602 CNRS
BlaisePascalUniversity
Aubière,France
samuel.gidel@lasmea.univ-bpclermont.fr
Abstract–In urban environments,pedestrian detection is
achallenging taskforautomotive research,wherealgo-
rithms sufferfromalack ofreliabilityduetomanyfalsede-
tections.Thispaperpresentsamultisensorfusion method
based on a stochasticrecursive Bayesian frameworkalso
called particlefilterwhichfusesinformation fromlaserand
videosensors toimprove theperformance ofa pedestrian
detection system.Themaincontributionsof thispaperare
first,theuseofa non-parametricdata association method
in orderto betterapproximatethediscretedistribution and
second,themodeling of thelikelihood function with a mix-
tureofGaussian and uniformdistributionsin ordertotake
into accountall theavailableinformation.Simulation re-
sultsaswell as resultsofexperimentsconducted on realdata
demonstratethe effectiveness of theproposed approach.
Keywords:Particlefilters,kerneldensityestimation,laser-
scanner,videocamera,sensor fusion,likelihood computa-
tion.
1Introduction
Currently,morethan 8,000 vulnerableroad users,pedes-
triansand cyclists,arekilledevery yearintheEuropean
Union.Accidentstatisticsindicatethatdespiterecentad-
vancesinsafety duetotheintroduction ofpassivesafetysys-
temsand tighterpedestrianlegislations,pedestrianaccidents
still represent thesecond largestcauseoftraffic-relatedin-
juriesand fatalities[1].So,thepedestrian detection be-
comesanessentialfunctionalityfor futurevehicles.This
issuetakesplace inthe contextoftheLOVeProject(Soft-
wareforvulnerablesobservation)whichaimsat improving
roadsafety,mainlyfocusing on pedestriansecurity[2].
Forabroadreviewofthevarious sensorsusedforpedes-
trian detection,one canconsult [3]and [4]wherepiezo-
electric,radar,ultrasound,laser rangescannersensorsand
camerasoperating inthevisibleorintheinfraredarede-
scribed.Using videosensorstosolvedetection and identifi-
cation problems seemsnaturalatfirst,giventhe capacity of
thistypeofsensorto detect/analyze thesize,theshape and
thetextureofapedestrian.Many methodsto detecthuman
beingshavebeen developedincomputervision based on
monocularorstereoscopicimages[6].However,thestrong
sensitivitytoatmospheric conditions,thewidevariability of
humanappearance,thelimitedapertureofthis sensorand
theimpossibilityto obtain directand accurateinformation
concerning depth havegivenrisetoaninterestforthedevel-
opmentofadetection method starting fromanactivesen-
sorlike a radaroralasersensor.The ability oflasersys-
tembased pedestrian detection tocountand track hasbeen
proved,eveninthe caseofavery high-densitycrowd[7] [8].
However,theobviouslimitationsofthis sensor (no informa-
tion aboutshape,contour,texture,colorofobjects),its sen-
sibilitytoatmospheric conditions suchasrainand fog and
thefrequentocclusionsbetween objects,requireto devise a
method oflaser/camerafusion toimprove a pedestriancol-
lision avoidance system.
Astudy ofsensor-based pedestrian detection,presented
in[5],indicatesthat thelaserscannercooperation withcam-
erasappearsto be a good solution to develop.So,theprob-
lemishowtocombinethediverse and sometimesconflict-
ing amountsofinformation inthebestmanner,to outper-
formthebestresultsexpectedfromtheuseofasinglesensor
technology.
Themain difficulty ofdatafusion liesinthe association of
thenewobservationscoming fromdifferentsensors.Thus,
two distinctproblemshaveto bejointlysolved:data associ-
ation and estimation.
The conventionalapproachesarebased on thelinear
Kalmanfilterand leadto data association suchas JPDAF
(JointProbabilisticDataAssociation Filter) ortheMHT
(MultipleHypothesesTracker) which differintheirasso-
ciation techniquesbutwhichall sharethesameGaussian
assumptions[9].Unfortunately,tracking pedestrian does
notcopewithlinearmovementand Gaussian noises.Un-
dersuchassumptions(stochasticstate equation and non-
Gaussian noises),particlefiltersareparticularlyappropri-
ate[10].Inthispaper,anovel laser/camerabasedsystemis
presented,thataimsatreliability,real-timemonitoring and
tracking multiplepeopleinan urbanenvironment.
Thisarticleisorganizedasfollows.InSection 2 the ap-
12th International Conference on Information Fusion
Seattle, WA, USA, July 6-9, 2009
978-0-9824438-0-4 ©2009 ISIF 626

proachintheLOVeprojectframeworkisexplained.Then,
thesystemand thesensorsused by Renault manufacturerare
described.Section 3 presentsanon-parametric approachin
amultisensor fusion framework.InSection 4,simulation
and experimentalresultsarepresentedto demonstratethe
effectiveness oftheproposedapproach.Finally,the conclu-
sion ispresentedinSection 5.
2Overview
2.1Ourapproach
Themultisensoroutdoorpedestrian detection systemhas
been designedtofit thetechnicalspecificationsdefined
withintheLOVeProject[2].Thepurposeisto develop
safe and reliablesoftwarefortheobservationsof"vulner-
ables".Howeverthis softwaremustallowafast indus-
trialexploitation (validatedsoftwaremodules,standardized
and portable).Inthiscontextofstandardization,theLOVe
projectspecificationsprovidedalistofcommon inputand
outputforall thesoftwareblocks.Inthisoutput list,thede-
tected pedestrianset isdefined by Zj
k=(z1
k,...,zNz
k),with
Nzthenumberofobservationsat instantk,and isasso-
ciatedwitha"Detection Overall Rate"(DOR)defined by
γ
j
k=(
γ
1
k,...,
γ
Nz
k)with(
γ
j
k∈]0,1])assessing itsreliability.
In ordertotrack pedestriansfromamultisensor frame-
work,aSequentialImportance Resampling ParticleFilter
(SIRPF)isproposed.It isbased on anon-parametric ap-
proach using theparticlesetand all theDORstocompute
probabilitiesforeach data association.Moreover,these
DORscan beincludedinthelikelihood function withtra-
ditionaluncertaintyassociatedateach detection.So,amix-
tureofGaussianand uniformdistributionsisproposed.The
problemishowtocombineinthebestmanner,thediverse
and sometimesconflicting information provided by twosen-
sorsin orderto obtain better resultsthanwith only onesen-
sor.
2.2Vehicledescription
AnIBEOlasersensorand twocamerasensorsequiptheRe-
nault testvehicle.
TheIBEO ALASCA XTismountedinthe centerofthe
frontalarea ofthevehicle and twoSMALvideocamerasare
on top ofthe car forsimultaneousrecording ofthescene(see
Figure1).
3SensorDataFusion
Thetask ofsensordatafusion inautomotive applications
usesmultiplesensorstoconstitute anall-around detection
systemto overcomethedeficiency ofanindividualsensing
device.Many workshavebeencarried out tocombinere-
dundantand diversemeasurementdatafromseveralsensors.
Intheparticularcaseofpedestrianclassification,severalap-
proacheshavebeen proposed.Fourmainfusion architec-
turesareidentifiedintheliterature:
•serial fusion: thelaserscannersegmentsthescene and
then provides someROIs(RegionsOf Interest),which
Figure1:Location ofsensorsintheRenault testvehicle.
are confirmedtomatch pedestriansby meansofavi-
sion basedclassifier [11];
•centralizedfusion: themeasurementsfromthevarious
sensorsaremerged(associatedand tracked)inasame
centralblock[12];
•decentralizedfusion:eachsensorsystemdetects,clas-
sifies,identifiesand tracksthepotentialpedestriansbe-
forebeing mergedinatrack-to-trackfusion block[13];
•hybridfusion:availableinformation includesboth un-
processed datafromonesensorand processed data
fromtheotherone.[14].
Inthispaper,a centralizedfusion architectureischosen.
3.1SystemArchitecture
Pedestrian detection systemarchitectureis showninFig-
ure2.Thepedestrian detection isperformedinthelaser-
scanner [8]and videoimageframes[15].Acentralizedfu-
sion moduleisdevelopedwhosemaincontributionsare:
•anon-parametricdata association,
•amixtureofGaussianand uniformdistributionsfor
likelihood computation,
•the computation ofafusion confidence factor.
3.2SIRPF
Inthefollowing section,thetheory ofthesequentialMonte
Carlomethodsintheframework ofmultipleobject tracking
isbrieflyreminded.Formoredetails,thereadercanreferto
Doucet’swork[10].
Letusconsideradiscretedynamicsystem:
Xk=f(Xk−1)+Wk(1)
Zk=h(Xk)+Vk(2)
627

Filter update
(likelihood computation)
Confidence Factor Data Fusion
in Data Fusion
Tracks
Association
(at the same time)
Pedestrian
LaserscannerVideo source
Pedestrian
Tracking Tracking
Pedestrian
Detection Detection
Pedestrian
Input Vector according to LOVe specifications
Figure2:Multi-module architectureusing lidarand vision
information forpedestrian detection and classification.In
red,maincontributionsproposedinthispaper.
whereXkrepresentsthestatevectorat instantk.Noassump-
tion ismade about thetwofunctionsfand h,whereasVk
and Wkaresupposedto betwoindependentwhitenoises.
Particlefiltersprovide anapproximateBayesiansolution
to discretetimerecursiveproblemsby updating adescrip-
tion oftheposteriorfiltering densityp(xk|z1:k).Thisapos-
terioribelief representsthestateinwhichtheobjectsare.
Thepriordistribution oftherecursiveBayesianfilter
p(xk|z1:k−1)isapproximated by asetofNsamples,using
thefollowing equation:
p(xk|z1:k−1)=1
N
N
∑
i=1
δ
(xk−xi
k)(3)
where
δ
isthediscreteDirac function.Thentheposterior
distribution p(xk|z1:k)can be estimated by:
p(xk|z1:k)=p(zk|xk)
N
∑
i=1
p(xk|xi
k−1)(4)
Thisapproachcan beimplemented by abootstrapfilterora
Sampling Importance Resampling (SIR)filter.
3.3Non-parametricdata association
3.3.1Introduction
Non-parametricmethodsallowtotakeintoaccount thesam-
plesand theirspace distributionsinthespace parameters.
LetNxbethenumberoftargetstotrack.Thisnumber
isunknownat instantk.Inthispaper,multi-target track-
ing consistsinestimating thestatevectorobtained by con-
catenating theNxvectorofall targets.ThevectorXk=
{x1,l
k,...,xNx,l
k}N
l=1isgiven by thestate equation (1)decom-
posedinNxequations:
Xk=Fk(Xk−1,Wk)(5)
whereNisthenumberofparticles,and thenoisesWkare
supposedto bespatiallyand temporallywhite.Theobserva-
tion vectorcollectedat timekisdenoted by Zk=(z1
k,...zNz
k),
withNzthenumberofobservationsdeducedfromthepro-
cess:
Zk=Hk(Xk,Vk)(6)
Once again,thenoisesVkaresupposedto beindependent
whitenoises.
The association matrixAkisintroducedto describethe as-
sociation betweenthemeasurementsZkand thetargetsXk.
Anon-parametricframeworkischosenin ordertoestimate
the association matrixAk.
Twotechniquesmakeit possibleto generate a succession
ofareaswhichsatisfy good conditionsofestimation:
1.by fixing thevolumeofthe area like a function ofn
samples,forexampleVn=1
√n.
It isthe"ParzenWindow"method.
2.by adapting thesize ofthe areaswithsamplenumbers
knfixedaccording ton,forexamplekn=√n.
It istheKnearestneighborsmethod.
Inthispaper,the"ParzenWindow"method ischosenin or-
dertoexploit theKernelDensityEstimation (KDE)allow-
ing toextrapolatethedataon the entirepopulation.Finally,
aBernoulli variablewh∈{w1,w2}isalso defined,given by
wh=w1ifthe associatedevent isclassifiedasfused dataor
wh=w2inall othercases.
3.3.2Parzenassociationforparticlefilter
Anapproachisproposedto buildanon-parametricmodel
based on kernelfunctionsallowing asmartselection ofthe
mostpertinentdatafusion fromalikelihood analysisfunc-
tion.Thismethod isnotsupervised,so no priorknowledge
isrequiredto process datafusion.Thelikelihood function
p(zj
k,xi
k|wh)representstheprobabilitythata2Dparticlebe-
longstothefused data.Thelikelihood p(zj
k,xi
k|wh)will be
modeled on aParzenwindow whichcalculatesthedistance
betweenan observation zj
klocatedintheimage and all its
neighborsxi,l
ksuchas:
p(zj
k,xi
k|wh)=1
N
N
∑
l=1
ϕ
(zj
k,xi,l
k)(7)
628

where
ϕ
(zj
k,xi,l
k)isthekernelfunction whichallowsto
modifytheinfluence zoneofobservation withitsneighbors.
AmixtureofGaussianand uniformdistributionsisusedin
ordertomergeinasamedistribution all theinformation
availablefrom mono-sensoralgorithmoutputs.
ϕ
(zj
k,xi,l
k)
isgiven by:
ϕ
(zj
k,xi,l
k)=(1−
γ
j
k)·U(−1
γ
j
k
,1
γ
j
k
)+
γ
j
k·exp[−
λ
c·dc(zj
k,xi,l
k)]
(8)
Theparameter
λ
cpermitstoadjust theweights.Thegener-
alizedsquaredinterpointdistance dcisdefined by:
dc(zj
k,xi,l
k)=(zj
k−xi,l
k)Σ−1
ϕ
(zk
k−xi,l
k)T(9)
withthe covariance matrixΣ
ϕ
,result ofthesumbetween
the covariance matrixΣSP2given by thetracking algorithm
(representing thevariance on pedestrian position)and the
measurementnoise covariance matrixR:
Σ
ϕ
=ΣSP2+R(10)
The2Dparticlexi,l
k∈w1having thehighestprobabilityis
chosen by themaximumlikelihood estimatorsuchas:
Ai,j
k|(i,j)=argmax
(i,j)
(p(zj
k|xi
k)|w1)) (11)
3.3.3Likelihoodcomputation
SIRPF allowstoapproximatethefiltering distribution
p(xk|z1:k)by aweightedsetofNparticles.So,thedata as-
sociation abovebeing known,thenextstepconsistsincom-
puting theweightsofall theparticlesbelonging tothe as-
sociated gravitycentergiven by (11).Thus,theweight list
Li,j
kiscalculatedfromamixtureofGaussianand uniform
distributions(see Figure3)in orderto keepall theinforma-
tion used during thedata association step.Li,j
kisdefinedas
follows:
Li,j
k={
ϕ
(zj
k,xi,l
k)}(12)
Finally,theweightsarenormalized beforetheresampling
stage.Thisalgorithmis summarizedinAlgorithm1.
−1.5 −1 −0.5 00.5 11.5
0
0.2
0.4
0.6
0.8
1
X [m]
Probability density
−1.5 −1 −0.5 00.5 11.5
0
0.2
0.4
0.6
0.8
1
X [m]
Probability density
Figure3:AnexampleofamixtureofGaussianand uniform
distributions,within bluetheuniformdistribution,in green
theGaussian distribution and inredthedistribution mixture.
Ontheleft,
σ
=0.15 mand DOR=0.95.Ontheright,
σ
=0.15 mand DOR=0.55.
3.4Computationof the confidence factorin
datafusion
All currentlytracked objectsaretestedto determineifthey
areornot theresult of fusion betweenlaserand video data.
Forthispurpose,eachtarget isevaluated by computing its
Confidence FactorinDataFusion (CFDF).
Three criteria constitutetheCFDF: theConfidence inthe
AgeofTrack(CAT),theDetection Overall Rate(DOR)and
theSensorFusion Rate(SFR).TheCATallowstoevaluateif
thepedestriantargethasbeentrackedforalong timeornot.
TheDOR,provided by mono sensoralgorithms,istherateof
confidence that thedetected object isactuallyahuman.The
SFR allowstoevaluateifthetrackistheresult ornotofdata
fusion betweenlaserand video pedestrianmeasurements.
TheCATand theSFR are computedfromaGaussian dis-
tribution:
CAT(t)=(1
σ
o√2
π
exp[−1
2(t−
µ
o
σ
o)]0<t≤
µ
o
1t>
µ
o
(13)
where
µ
orepresentstheminimumtimeoflifeofatrack
withoutobservation,
σ
oallowsto decreasemore and less
quicklytheCATand tisthe ageofthetrack.
SFR(x)=1
σ
f√2
π
exp[−1
2(x−
µ
f
σ
f
)](14)
Where
µ
frepresentsthetheoreticalratio betweenthenum-
beroflaserdata and thenumberofvideo data,
σ
fallowsto
decreasemoreorless quicklythesensor fusion rate and xis
theratio betweenthenumberoflaserdata and thenumber
ofvideo data.
Finally,thefinalresult isgiven by:
CFDF=CAT+SFR+DOR
3(15)
4Experiments
This section presents simulationsand experimentswhichal-
lowto validatethe algorithmoflaserand video datafusion.
4.1Simulations
Thegoalofthesesimulationsistoshowhowthe algo-
rithmofdatafusion improvesthepedestriantracking sys-
tem.First,astudy ofthisdata association algorithmispro-
posed.InFigure4,ascenariowithtwo pedestriansde-
tectedfromlaserand video datawasgenerated.Therele-
vance ofadata association based kerneldensityestimation is
demonstrated herewhenseveralmeasurementsfromdiffer-
entsensorscan be associatedwiththesametrack.Second,
astudy oflikelihood computation isproposed.InFigure5,
a cloud of randomparticles(red points)wasgenerated.The
particles(bluestar) representing the cloud centerwerese-
lectedasmeasurements.According tothelikelihood (see
Figure3),theweightsare computed(green points).With
thesameuncertaintyconcerning theposition,theresultsare
differentaccording tothe"detection overall rate".Thislast
629

Algorithm1Non-parametricDataFusion
1.Setk=0,generateNsamplesfromeachmeasurement
j=1,...,Nz,i.e{xj,l
0}N
l=1={x1,l
0,...,xNz,l
0}N
l=1where
xj,l
0=p(X0j).
2.ComputethematrixAkforall measurementsand
targets(Nz,Nx).
if(Ak≤
α
)with
α
=thegatethreshold
Ai,j
k=p(zj
k,xi
k|wh)wherep(zj
k,xi
k|wh)isthe associa-
tion probabilityforhypothesisiusing Nparticlesac-
cording toequation (7).
else
Ai,j
k=0then{xi,l
k}={xi,l
k−1}go toitem5.
3.Computetheweightswi,l
k=Li,j
k(12)and normalize,
i.e,wi,l
k=wl
k
∑N
l=1wl
k
.
4.Generate a newset{xi,l∗
k}N
l=1by resampling withre-
placementNtimesfrom{xi,l
k}N
l=1,wherePr(xi,l∗
k=
xi,l
k)=wl
k.
5.Predict(simulate)newparticles,i.e,xi,l
k+1=f(xi,l∗
k,vk),
l=1,...,Nusing differentnoiserealizationsforthepar-
ticles.Computeforeachestimation itsCFDF(15).
6.Increasekand iteratetoitem2.
point isimportantbecause,when pedestrianclassification is
notreliable,morehypotheses(particles)should bekept in
ordertocorrectapossible errorinthestate estimator.
4.2Experimentsonrealdata
Variousresultswithreal laserand video datafusion arein-
troduced.Lidarand cameradata arenotgiveninthesame
reference frames.Thereference framerelatedtothelidar
waschosenfor fusion.TheSIRPF withParzenWindowas-
sociation wastestedinmany differentsituationson realdata
provided by Renault,theFrench vehiclemanufacturer.The
presentedscenarios(see Figure8,9and 10)includeseveral
pedestrians(>5)who appearand disappearinthesensor
area and showdifferentsituations suchasan urbanscene,a
semi-urbanscene,ora carpark.Thepedestriansmoveinall
directions.Thevehiclemovesataspeedranging from0to
50 km/h,whichallowstotest therobustness ofthismethod.
Usually,in pedestrianclassification framework,lidarmea-
surementscan generatefalsetracks[17].Theselidarmea-
surementsresult inmostcasesfromfixed objects suitable
foran urbanenvironment.Falselidarmeasurementscan be
assimilatedwithsecurity barriers,poles,trees,etc.Foreach
iteration,thenumberof falsedetectionsisobtained by cal-
00.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
X [m]
Y [m]
Particles of object 1
Particles of object 2
laser data
video data
data5
O1: position uncertainty : +/− 5 cm
pedestrian confident rate : 0.55
O2: position uncertainty : +/− 70 cm
pedestrian confident rate : 0.95
O3: position uncertainty : +/− 55cm
pedestrian confident rate : 0.89
Figure4:Data association fromseveralsensors.Here,ac-
cording tothenearestneighborcriterion,O1would be as-
sociatedto object1and O3to object2,whereasthe correct
association isgiven by theParzenalgorithmi.eO2with ob-
ject1and O3with object2,whileO1isinrealityapole.
−1.5 −1 −0.5 00.5 11.5
−1.5
−1
−0.5
0
0.5
1
1.5
X [m]
Y [m]
−1.5 −1 −0.5 00.5 11.5
−1.5
−1
−0.5
0
0.5
1
1.5
X [m]
Y [m]
Figure5:Exampleoflikelihood computation on a cloud
ofparticlespresentedinFigure3.Inleft,
σ
=0.15 mand
DOR=0.95 and inright,
σ
=0.15 mand DOR=0.55.
culating theratio:
rate_of_false_detection =NT−NP
NT
(16)
withNTthetotalnumberofdetectionsand NPthenumberof
detected pedestrians.Therateofdetection ofpedestrian(s)
isgiven by calculating theratio:
rate_of_pedestrian_detection =NP
NP_VT
(17)
withNP_VTthenumberofpedestrianswho are actuallyin
the area observed by thesensors.Table4.2showsthe ad-
vantageofdatafusion tosignificantly decreasethenumber
of falsedetectionswhenasinglelidarorasingle camera
areused.Itcanalso benoticedthat therateofpedestrian
detection ishigherafterdatafusion.Toconclude,astudy
illustrating theresultsoftheCFDFmoduledepending on
timeisproposed,where eachtrackisassociatedwithadif-
ferentcolor (see Figure6and 7).This study isconducted on
amulti-pedestrianscenario presentedinFigure9.
630

Table1:Rateof false and correctdetectionsforthescenarios
presentedinthe article:when onlylidaroronlycamera are
used,and afterdatafusion.
LidaronlyCameraonlyAfter fusion
False
Detection
Rate
0.536 0.274 0.108
Pedestrian
Detection
Rate
0.916 0.702 0.928
0 10 20 30 40 50 60 70
−4
−2
0
2
4
Y [m]
0 10 20 30 40 50 60 70
0
5
10
15
20
iterations
X [m]
Figure6:Result ofmulti-pedestriantracking on Xand Y
positions.Measurements(videoand lidar) are constantly
represented by graycircles.Eachtrackisrepresented by a
differentcolor.
0 10 20 30 40 50 60 70
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
iterations
CFDF
Figure7:Result ofmulti-pedestriantracking withCFDF.
EachCFDFisrepresented by astarofadifferentcolorwhile
eachDORisrepresented by adotofadifferentcolor.
5Conclusions
Thispaperpresentedamultisensorpedestrian detection
system.The centralizedfusion algorithmisappliedina
Figure8:Detection exampleina cross section aftera cen-
tralizedfusion fromlidarand videoimagedata.Thered
dotsrepresent lidardetection and thebluerectanglesrepre-
sentcameradetection.Theyellowrectanglesaretheresults
provided by thedatafusion module.
Figure9:Detection exampleina carparkaftera centralized
fusion fromlidarand videoimagedata.Thered dotsrepre-
sent lidardetection and thebluerectanglesrepresentcamera
detection.Theyellowrectanglesaretheresultsprovided by
thedatafusion module.We canalso notice a correctpedes-
trian detection atadistance up to 25 meters.
Bayesianframework.Indeed,in ordertotrackmore eas-
ily pedestriansrandom movementswhichcaninclude abrupt
trajectorychange,aSIRPF waschosen.First,in orderto
takeintoaccount theunspecifiedcharacterofthedistribu-
tion ofparticlespredicted by theSIRfilterand all theDOR
given by mono sensoralgorithm,adata association based
on kerneldensityestimation wasused.Second,alikelihood
based on amixtureofGaussianand uniformdistributions
wasused; thusit waspossibletotakeintoaccountmorepre-
ciselyall the availableinformation relatedtotheuncertain-
tiesoflaserand videomeasurements(uncertaintyconcern-
ing boththepedestrians’position and classification).Ex-
631

Figure10:Detection exampleina cross section aftera cen-
tralizedfusion fromlidarand videoimagedata.Thered
dotsrepresent thelidardetection and thebluerectanglesrep-
resent the cameradetection.Theyellowrectanglesarethe
resultsprovided by thedatafusion module.Pedestriansin
variousorientationsaredetected.
perimentalresultson simulated data aswell ason realdata
demonstratedthe effectiveness ofthisapproach.
Thenextstepistotest this systemon moredatasequences
to be abletocharacterize it intermsof falsepositive and
correctdetection rates.
References
[1]http://www.euractiv.com/en/health/road-safety-
pedestrians/article-117530
[2]http://love.univ-bpclermont.fr/
[3]T.Gandhiand M.M.Trivedi,"PedestrianCollision
Avoidance Systems:aSurvey ofComputerVision based
RecentStudies",inProc.IEEE IntelligentTransporta-
tion Systems2006,Sept2006,pp.976-981.
[4]F.Buand C-Y.Chan,"PedestrianDetection inTransit
BusApplication:Sensing Technologiesand SafetySo-
lutions",inIEEE IntelligentVehiclesSymposium(IV),
LasVegas,June2005.
[5]D.M.Gavrila,"Sensor-basedPedestrianProtection",
IEEE IntelligentSystems,vol.16,NR.6,pp.77-81,
2001.
[6]F.Arnell,"Vision-BasedPedestrianDetection System
foruseinSmartCars",Master’sThesis,Stockholm,
Sweden,2005.
[7]X.Shao,H.Zhao,K.Nakamura,K.Katabira,
R.Shibasakiand Y.Nakagawa,"Detection and Track-
ing ofMultiplePedestriansby Using LaserRangeScan-
ner",IEEE/RSJInternationalConference on Intelligent
Robotsand Systems,SanDiego,USA,2007.
[8]S.Gidel,P.Checchin,C.Blanc,T.Chateauand
L.Trassoudaine,"PedestrianDetection Method using
aMultilayerLaserscanner:Application inUrbanEn-
vironment",inProc.IEEE/RSJInt.Conf.on Intelligent
Robotsand Systems(IROS),2008.
[9]Y.Bar-Shalomand X.R.Li,"Multitarget-Multisensor
Tracking:Principlesand Techniques",ISBN0-
9648312-0-0,1995.
[10]A.Doucet,S.Godsill,and C.Andrieu,"Onsequential
MonteCarlosampling methodsforBayesianfiltering",
Statisticsand Computing,vol.10,CA,no.3.pp.197-
208,2000.
[11]M.Szarvas,U.Sakaiand J.Ogata,"Real-timePedes-
trianDetection using Lidarand ConvolutionalNeural
Network",inProc.oftheIEEE IntelligentVehicles
Symposium(IV),Tokyo,Japan,2006.
[12]D.T.Linzmeier,M.Skutek,M.Mekhaieland
K.C.J.Dietmayer,"APedestrianDetection System
based on Thermophile and RadarSensorDataFusion",
8thInternationalConference on Information Fusion,
Philadelphia,USA,2005.
[13]C.Blanc,L.Trassoudaine and J.Gallice,"EKFand
ParticleFilterTrack-to-TrackFusion:aQuantitative
Comparison fromRadar/LidarObstacleTracks",8thIn-
ternationalConference on Information Fusion,Philadel-
phia,USA,2005.
[14]G.Monteiro,C.Premebida,P.Peixotoand U.Nunes,
"Tracking and Classification ofDynamicObstaclesUs-
ing RangeFinderand Vision",inIROS2006 -Work-
shop on "SafeNavigation inOpenand DynamicEnvi-
ronments-AutonomousSystemsversusDriving Assis-
tance Systems",Beijing,China,2006.
[15]L.Leyrit,T.Chateau,C.Tournayre and J.T.Lapreste,
"Association ofAdaBoostand KernelBasedMachine
Learning MethodsforVisualPedestrianRecognition",
inProc.oftheIEEE IntelligentVehiclesSymposium
(IV),Eindhoven,Netherlands,2008.
[16]L.Trailovic and L.Y.Pao,"Position errormodeling
using Gaussianmixturedistribution withapplication to
comparison oftracking algorithms",inAmericanCon-
trolConference,Denver,USA,2003.
[17]S.Gidel,P.Checchin,T.Chateau,C.Blanc and
L.Trassoudaine,"ParzenMethod forFusion ofLaser-
scannerData:Application toPedestrianDetection",in
Proc.oftheIEEE IntelligentVehiclesSymposium(IV),
Eindhoven,TheNetherlands,2008.
632