Proceedings of the IRCTR Colloquium on Surveillance Sensor Tracking: Papers presented at the International Colloquium held in Delft, The Netherlands, 26 June 1997

(1)

NTRE FOR TELECOMMUNICATIONS-TRANSMISSION AND RADAR

, . . .. . . . DELFT UNIV ER SITY OF TECHNOLOGY - DEPARTM ENT OF ELECTRI CAL ENG I N EE R I NG

Proceedings of the IR CTR Colloquium on

Surveillance Sensor Tracking

(2)

Proceedings of the IRCTR Colloquium on

Surveillance Sensor Tracking

in cooperation with NC3A

Delft, 26 june 1997

~

i

'

TU Delft

Papers presented at the International Colloquium held in

Delft,

The Netherlands

,

26 june 1997

(3)

On the organisations

IRCTR

is established

as

a centre

of

expertise

in

the

fields

of

Telecommunications-transmission and Radar. IRCTR operates as a

non-profit making institute, based within the Faculty of Electrical Engineering of

the Delft University of Technology in The Netherlands. IRCTR is a project driven research

institute. The pre-competitive projects are supported by the NWO (National Science research

Council), the STW (Foundation for Technical Sciences), the Dutch ministry of Education,

Culture and Sciences, the GTI's and industries.

IRCTR provides the essential interface to

promote international research advancement.

At the IRCTR the fundamental and experimental research projects are being carried out in

four sectors

..

The research areas in some keywords

:

Sector Antennas and propagation:

time domain measurement, hybrid reflector systems

,

wide band antennas, modeling of propagation.

Sector Radar:

Radar System Design, Wide band radar, multi-parameterlDoppler polarimetric

radar, integrated radar communication.

Sector

Transmission:

Hybrid

multiple

access

schemes

,

broadband

multi-media

communications, wireless ATM, networking, smart antenna and coding

.

Sector Remote Sensing:

Study of the atmosphere, espe-cially on clouds. Physical parameters

in the scattering process of radar waves, extraction of these parameters.

The NATO Consultation, Command and Control Agency (NC3A) is resulting

from the merger of the forrner SHAPE Technical Centre (STC) and NATO

CIS

Agency (NACISA). NC3A's function is to provide scientific and

technical advice and assistance to the NATO commanders in matters

relating

to

land, air and naval forces.

NC3A is also a single unbiased acquisition agent in the

implementation of NATO C3 capabilities.

Programme committee:

M

.Desbois

(NC3A)

Prof. L.P.Ligthart

(IRCTR)

Prof. P.van Genderen (lRCTR)

Mrs W.L.M.van

der Voort (lRCTR-secretary)

The content of this publication has been reproduced directly from the material supplied by the

authors

.

(4)

Uitgegeven en gedistribueerd door:

Delft University Press

Mekelweg 4

2628 CD Delft

telefoon:

015-2783254

fax:

015-2781661

E-mail:

DUP@DUP

.TUDelft.NL

elP-GEGEVENS KONINKLUKE BIBLIOTHEEK, DEN HAAG

Surveillance

Surveillance Sensor Tracking - Delft

:

Delft University Press.

-

Illustrations.

ISBN 90-407-1488-6

NUGI 841

Trefw.:

Surveillance, Sensor, Tracking

Copyright

All rights reserved.No part of the material protected by this copyright notice mayhe reproduced or utilized in any forrn or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system,without pennission from the publisher:Delft University Press,Mekelweg 4,2628 CD Delft, The Netherlands.

(5)

Session 1: Multiple Sensor Tracking

(Chairman: P.van

Genderen)

1. Noncooperative Target Type Identification in Multi-Sensor Tracking

By T.Wigren (Celsius Tech, Sweden)

2. Multi Sensor Data Fusion

By J.N.Driessen (Hollandse Signaalapparaten, Netherlands)

3. Multi Sensor Data Fusion in ARTAS

By C.M.Rekkas (Eurocontrol

,

Brussels)

,

RA.P.Blom (NLR, Netherlands)

Session 2: Tracking Performance Assessment

(

Chairman: M.Desbois)

4. Wavelet Filters in Trajeetory Reconstruction from Multi-Radar Data

By S

.Mancini,

B.Lambin, P.Kerstens (Eurocontrol, Belgium)

5. Objective Quantitative Tracker Performance Analysis

B

y

R.A.Hogendoorn, B.H.L.Neven, J.Westland (NLR, Netherlands

)

6. Surveillance performance assessment: a method supported by simulation tools.

Application to sensor deployment analysis

.

(6)

NONCOOPERATIVE TARGET TYPE IDENTIFICATION IN MULTI-SENSOR TRACKING

Dr.Torbjörn Wigren

CelsiusTcch Systems AB. S-175 88

Järf

älla.

SWEDEN.Phonc+4 6 8 5 80 84 000

Abstract

The paper focuses on noncoopcrative largel type identification algorithms. that are being developed for CelsiusTech Systems mulli-sensor tracker (MST).This MST.which is briefly described in the paper. has been developed for a major NATO customer.

One main purpose of the paper is to prescnts details on the algorithm that processes and fuses kinemaIic type related information.The creation of cxperimental Ilight envelopes for each track and Iheir processing10obtain target type estimates are discussed. Expcrimental results on the performance obtained with the kinematic target type identification algorithm are given. A 72 target South Atlantic scenario with 9 target types was used toassess the performance.A prototype MST.with functionality for noncooperative target type identification. was used to conduct the expcrimcnts. The cxperimcntal MST was first run as an MRT with data from 4 radars and then as an MST with 3 additional airborne ESM sensors.

I. INTRODUCTION

In the past jammer bearings (ECM) { I

I

has been the major souree of passive information in command and control systems. This situation is currently changing rapidly with the appearance of highly accurate electronic suppon measures (ESM) I1

I.

This fact opens up ncw possibilities for combined kinematic multi-sensor tracking [ 2 land noncooperative target type identification. e.g.

I

3

I.

using a multitude of different infonuation sources. To exploit these possibilities CelsiusTech Systems has developed a prototype MST with fully integrated functionality for automatic and noncooperative target type idenlification [ 4

I

.

The system uses a combination of ESM sensor mcasuremcnts and kinematic Ilight envclope rclated estimates obtaincd from the kinemaIic tracker, 10 compute target type probabiIities for each track. Data is fused over time with Bayesian methods.An overview of the system is given and the use of target type information in the expertmental MST system is discussed.

The cxpcrimcntal MST system has been used in order to perform cxperimems with the target type identilication algeritluns.A major purpose has been 10 perform realistic simulations with a large number of targets in cluttered and jammed situations. The kinematic pan of thc cxpcrimental MST can then be cxpectcd to provide much more realistic inputs to the kinemalictarget typeidentilicalion al gorithms. as comparcd to stand alone trials. A 72 target scenario from the Falklands Islands is discussed in the paper. The cxpcrirncntal MST was lirst mn as a centralized multi-radar tracker (MRT) with data from 4 radars. After that, 3 airborne ESM sensors were added to illustrate possible improvements. The target types included the AVRO Vulcan (not operational today).the Sea Harrier. the A4 Skyhawk, the Dassault Super Ethcndard,the Pucara.the Mirage 11I and the Mirage 5.

see for examplc

I

5 ].

I

6

I

for descriptions of the aircraft:Intercept and auack missions were considered for the two lalter types. resulting in a total of 9 target types.

The paperis organized as fellows.Section 2 comains a brief description of CclsiusTech Systcms MST. to deline the system for which the target type idcntification algeritluns have been devcloped.This MST is available as a product today. Section 3 discusses the basic principles of the idcntification algorithms. considering also ESM data.The metbod for targettype identification with kinema tic type related data as input is presented in section 4. Section 5 describes the cxperimental MST system. the South Atlantic scenario. as weil as the results of the simulations. The conclusions appear in section 6.

2. TUE OPERATIONAL MST

Before the details of the target type identilication algorithms are described.it is instructivc to present an overview of the operational MST system (that does not have target type identilication included today).A more detailed description is available in [ 4

I.

Briefly the operation can be cxplaincd as follows.sec Fig.I.

(7)

Fig.I:T he main blocksof thc MST. Strobes and tracks carry both kinem ati c and type relat ed intermation in the experime ntal MSTwith non coop cr ativetar gettype identification.

• Plots and strobes (bear in g only measur ements) are firstpreprocessed and collec ted in timebatch es. • The plots and the strobes are then sent to the

tracking kemel for assoc iatio n to existing trac ks.

Associated plots andstrobesare usedfor updat in gof thesetracks.

• Plots that are not associa ted totrack s aresent tothe plot handIer for initiationofncwtrack s.

Strobes that are not associa ted to trac ks aresent to the strobe handier for ini tiati on of new tracks. Bayesianalgorithms are used for trian gul ation and deghosting.

• The bias calculator co ntinuous ly computes bias compensa tion parameters from mea sur ements that are firmly and uniquely associated to high qualit y tracks.

• Tracks and crossings are sent for display.

TheMST makes full use of sensor oriented coordinate systemsfor maximum flexibility.Thismakes it possible to handle an arbitrary mix of activo and passive measurements at all times. which allows for very powerful sensor management functi on s. The MST auto matica lly processes dat a from the followingsensor types

• 2-0 and3-0 active rad ar. primarysurvei lla nce rad ar (PSR) and secondary surve illa nce radar (SSR). mech ani call y scanned ante nna (MSA) and electronica llysca nnedante nna (ESA). ground based andairborn e.

2

• 1-0and2-0 passi verad a r (jamstro bes).othe r rad ar properti es asdescribed abovc.

• 1-0and2-0 ESMstrobes.MSA and ESA.ground based andairborne.

All tracks are represent ed and propagated in a cartes ian. 3-di me ns iona l. eart h tan gential coordi na te svstem centeredsomewhe re inthe surve illa ncearea.The

-

track~

are tran sform cd to map coord inates for display. The associatio n of data to tracks is per fo rm ed in the

measur em ent space to allow a handling of passive measurement s without ran ge informat ion ina coherent manner.The trac ksare thustran sform ed to (poss iblya lower dimension al) measur em ent space wherc a co nvc ntiona l multipl e hypoth esis maximum likel ihood assoc iatio n ispcrform ed.see

I

2

H

4

I

for details.To improve the perform an ce. scan to scan mem o rv is introduced in the associatio n process by the use of multiple hypothesistra cking (MHT).see

I

3

I

.

The MST described here implomeuts a variant of MHT tha t is sometimes dcnotcd track oriented MHT . cf.

I

7

I

.

All associations with a sufficiently high qualit y, togeth er with a coas ting alternative. are then retain ed. The resulting hypotheses are used in orde r to form alterna tive tracks.foreachsystem track.The probabili ty that eachalterna tive is correct. conditionedon the fact that there is onlyone tar get. is updated with Bavcsian techniques. Hypoth eses are removed when the'v fall

(8)

lar nd lil. m. .ks 'he :he ive ent "a a od To is of ST is <\11 Ier 'he rm ity act ian 'a ll

The tracks are also updated in the mensurementspace of each sensor. using extended Kalman filter (EKF) techniques.

The MST makes full use of interactive multiple model (IMM) filtering [ 8

I. [

9

1

for accurate modeling of maneuvering targets. The movement models are described in detail in

I

4

I

.

To summarize. the four modes of the IMM filter are

• uniform horizontalflight. • speedchanges.

• slow coordinatedtums. fast maneuvers.

This adva nced filtering technique allows the MST to securely track also highl ymaneuvering fightersin heavy c1utter. Functionality for tracking of tactical ballistic missiles (TBM's) is available as an option [10

I

.

The initiation of tracks is treatedseparatelyfor active (plots) and passive (strobes) measurements.The active initiation reli eson multiple hypothesi stechniques.from scan10scan.The automatic passive initiation is much more elaborated. Briefly, it relies on triangulation of strobe tracks to form crossings. The quality of each crossing isthen evaluated with very advanced Bayesian techniques. The reader is referred to

I

4

I

for further detail s.Adaptivec1utterdensi ty mapsareautomatica lly updated in the system to support the track in itia tion process.

As in all successful centralized tracking systerns, a weil design ed bias calculator is crucial for good performance.CelsiusTechSystemsis using a design that automatica lly update s up to I I parameters per sensor. Furthe rinformationon the bias calculator isavailablein 111].

The MST is implemented in a COTS workstation of thetype IBM RS/6000Model390.The tracker is written in ADA and the AIX operatin g system is used. The communication withthe otherparts of the C~syste m is via TCP/IP on Ethernet. The MSTsystemhas 8 sensor channe ls. The maximum capacity is more than 250 track s and 250 measurements per secondcontinuous ly, 500 measurements peak.The capacity can be enhanced bya factor of4as an option. simply by using higher performan ce hardw ar e. Processin g power can also be distributed between computers. which gives full flexibilityin terms of capacity.

J. TARGET TYPE ESTIMATION PRINCIPLES Thereexists a number of techniquesthat are potentially useful for noncooperative target type ide ntification. Apa rt from the classical Bayesian methods.] 2

I. [

3

I.

Dempster-Shafer evidential reasoning.]3

I. I

12

I. [

IJ

I

has found widespread applications. The main advantage of the Dempster-Shafer method is c1aimed 10be its abilityto modelignorance.lt iswell documentedinthe literature that the modeling of ignora nce ca n be a sign ificant difficultywhen Bayesianmethodsare used.

A number of requirements on the design of the target type estimation algorithms wereset upat thestartof the work.Theseincludedthe following.

• The target type estimator should beflcxibleenough to allow the integration of different types of type related data. As a start , ESM data, experimental !light envelopes and direct target type observation s wereto beconsidered.

• The targettype estima tesshould be used throughout theMST and not onlybedisplayed to theoperator. Performance gains were expected in the deghosting. the active track initiation,the data associationand in theMHTalgorith ms.

• The computational complexity and the sicrage requirementsshould be kept low to allow real time implementation without reducingthecapacity of the MST significa ntly. A recursi ve in time ..filter" solution was preferred.

• The algorithms should be based on safe prior knowiedge. Assumptions on tactic al beh avior should be avoided.

CelsiusTech Systems solution is to use c1assical Bayesian methodology togeth er with certa in approximations to obtain a low complexity recursive target type filter. The target type filter estimatcs the probabilities of the target types ina predefined set of targettyp es. The method makes the integration in the MST straightforward since the data processin g in the MST builds on Bayesian methods throughout. Approximations and certain renonnalizations result in a recursive algoritlun that significantly reduces the co m plexity inherent in the full implementation of Bayesian algorithms.New data types are integrated into the system by calculation of the likelihood of the measuremcnt conditioned on the tar get type and the earlier measurements.Itis the taskofthe design erto use sensorknowledgeto definesuitable approximations that allow a low complexity likclihood computation. yct

capturing the main propertiesof the new datatypes.In this way CelsiusTech Systems has been able to successfully integrate ESM dat a. experimental !light envclopes and direct type observations in a sing le filteringprocesswhere all dataare fused overtime.The only (safe priors)that are used are the flight envelopes of each target type. the possible eruitter modes/radar typesfor eachtargettype and an assum edprobability of correct dereetionof emitter modes for eachsensor.

(9)

To formalize the algorithm. introduce the quantities

l

"

I

N

,

{

}N

U

l

'

I

N

,

I . M . K, . zr(l/)· ZM(I ). z,,(I) .

r ,::.1 ) )=1 k=J m n

Z(lq) and Z(tq)'They denote the set of target types. the set of emitter modes/radars. the set of kinematic property estimates. the direct type measurement, the emitte r mode/radar type measurcmcnt , the kinematic measurement. an arbitrary measurementand the set of all measurementsobtained so far. Now observe that the targ et types (correctly defined) form a mutually exclusive and exhaustive set. Hence Baye s rule and the total probabilitytheoremgive I91

for i=l...Nr' Equation ( I ) would be rccursive,

whereit not for the quantity Z(Iq_l) in the likclihood p(tn :z(t.)17;.Z(I._I»)' The task of the designer is hence 10obtain suitable approximations so that the infinitedepend enee on all prior data disappcars. When this goa l is achieved. ( I )can be iterated over time sta rting with the prior target type probabilities

p,

0.

Note thefollowing

• Dataarefusedover timeand condensed in the largel typeprobabilities(the statcs).

• Different types of data are fused into the stales of one filter (for cach track).

• The integration of new types of data only involves the calculation of the likelihood of ( I ).

Note that sepa ra te time indices for each data type is used.Notealso thatall quantities of ( I ) are evaluated at time tq (the time of the latest measurement).Thelef! hand side is the updated state,while the state appearing in the right hand side is the statepropagatedto Iq. Havingdescribed the update step of the filter.it remains to find a procedure to propagate the target type probabilities belween measurements.To define such a procedure.note that

• targettype information is reduced in quality between mcasurements (as is all other informaticn).

• The effect of measurementerrors should disappear as thetime mcasuredfrom the error incrcases. To reflect this. the target type probabilities should collapse towards p'°.i

=

1,.... Nras time incrcases.

+(p(t._₁.7:IZ(t._I»)-p'0)e-r/,-I,, )/, (2)

,

Proceeding with the kinematics. it is clear that the "measurcmenr" of kinematic properties must be extracted from the kinematic state estimates of the track. or directly from plots/strobo s. In general these property measur ement s will thereforebeestimate sand they should be modeled as continuous stochastic variables. Typically these measurements(Ihe cxpcrimenta l Ilight envelope ) will ( I)

p(tq.z(tq)1

T. .Z(I

q_l))p(tq.7;IZetq_I») N,

L

p(tq. z(lq)1

T. .

zo

.,

»)I'(tq.

7 ;

I

Z(Iq_I»)

''''1

(10)

( 4)

Here

I

j

.

",

n

and I

r

.

ma

x

are subsets of the full fligh t

envelope of

T,

.

correspond ing to the selected measurements.A division into componentsfor different

attitudes and speeds is naturally incorpo rated into the vector formalismof(4).

Since Z,(I,,) is modeled as a continuous stochastie

variabie. the likelihood needs 10 be evaluated by

integration.The stati sti csof the estim ator corresponding to the forma! kinemati c measur em en t is needed for a

strict evaluation of the quant ities of inte rest. It is not

likely that these quaru ities can be easily computed.

However.if the following assumption isintroduced.the

problem is greatlysimplified.

envelope of target type 7; (with the formaI measurements defined by ( 3

»

at time InCan be

expressed as

A5) The estimators of ( 3 ) are consiste nt and the

supports of theirjoint probabilit ydcnsity functions are negligible as comparedto 1i'

This is sufficient forevaluationof the likelih ood.A3).

A.J). A5) ( I). ( 3 ).(4) andthcfact that allkinematic property variabl es are common to all types of tar gets

give ([ min

t.

(

X(S») ]

(

[Î

(I )] Z,(I,,)=C '''.

-".

n .

'"

C •

n.

n "

(

3 )

max,.,_-_~ km..

,

(

x

(

s»

)

1 (I) max " have to be related to prior infonnation on the flight

envelope.Someexa mples are then estimates (foreach

track) ofmaximumaltitude.minimum/maximumspeed.

maximum c1imbrate.maximumtransver sal acccle ration.

maximumlongitudinal accelera tionasweilasmaximum

dista nce from the base (to be compared to prior range

information). In view of the above the following measurement model is conside red

Here ~(s)denot esthe estima ted kinemati c sta tevector

ofthe track at time

s . K",

Jf(s» )

denotesthe vector of

(momentary) kinematicproperty estimat esof minimum

quantities. fmaJf(s») de notes thevecto r of(momentary)

kinematic property esti ma tes of maximum quantities.

The min . and max . operations corresponds to the

3~1" .r...t.

buildup of experimentalflightenvclopes for each track.

Theexperime ntal flight envelopesare divided into the quantit ies

I

rmn(In) and

I

max(I.,) as shown by ( 3 ).

C

denotes an indicator function indicatin g consistency

with a prior flight envelope. The measurement time I" can beselected bytheuser. One nalural choice is to

formally ..measure" al eac h time a track is updated. This isthe choice usedinthecxperimc nts ofsection5.

The experimental flight envclopes are conveniently

impIemented as tables where values are filled in

whenever ncw extreme values appear. These tables are

typicallyaltitudeand speed dependent.Finally,note that

a slidingwindow can repl acetheinfinitememoryof the measure ment (3 ) if thisispreferred .

The next step is then to computc the likelihood. No measurement history is retained in the likelihood

computation in thiscase.The motivation for this is that Iittle would be gained since the history is anyway

conde nsed in the experime ntal flight envelopes (the

fon nal measurement ).Thisisexpressed in

is 11 :s n n d e n IS Ir Ie 1. n n

:v

Ir e I. ,[ .r

e

f

In order 10 cornpute the likelihood the concept of

consiste ncy with thetrue flight envelope of a target type

needsto befonnali zed.

The flightenvelopeis a vector ofallowed interval sof

variation for the kinem atic variables related to the

partic ular targettype. If x(s) denotes the true stat eof

the target and if the flight envelope of target type

T,

is

denoted by 1, .cons iste ncyof thetargetwith theIlight

1

1.

[

~:~:~,~]

E

[

~

:

:::]

=

O

.

[~

"'n(l,,)]

,,[

~

r

,

n

.n]

fmax(/,, ) r.max

(11)

for i=1,...•NT' Here

p()

denotes the probability

densityfunction and à~ )isDirac'sdelta function. The recursions (1) are thus updated with I ifthekinematic property measurement is consistent with the flight envclopeand0otherwise.A smooth transition band can be introduced if amore smooth decisionispreferred.A lineardecrease from 1 to0isused in the experimental

system described here.

Apart from providing informationon the targettypeof

atrack.the kinematic information is usefulalso in other functions of the tracking system. 1tcan be used in a refinedtrack initiation procedure(activemeasurements on1y).Secondly.thepruningof MHT hypotheseswould benefit from kinematic type related information since inconsistentmaneuveringcould be detected inauniform marmer.

5. EXPERIMENTAL EVALUATION

The experimental MST system is mainl y used for development of new technology. The system is

implementedinC++ (VisualC++)on Windows NTand it is typicallyrun on Pentium PC"s. The system contains

virtually all functionalityof the full scaleMST that was described in section 2 of this paper. The main

exceptionsare thatatwo mode.constant velocity IMM filter with high/low process noise isusedand that the bias calculator is excluded.

Inaddition to the kinematictrackin gfunctionality.the

experimental system runs the algorithms for noncooperative tar get type ide ntification that are

described in thispape r and in

I

4

I

. '\

he system isthus capable of fusing (over time) ESM data (emitter modes/emitters), direct type observations and experimental flight envelope information. The system contains Windows basedsupport for definition of target pat hs. target characteristics. sensors. sensor

characteristics as weil parametersettin gs.Somemanual

commands area1so implemented.

The main purpose with the experimental MST

simulations that are presented here is to illustrate the possibilities that arise when new data like ESM is fully integrated in the tracking software. The purposeis notto present a maximally realisti c simulation from an operational point of view. Sensor data and flight

envelope data should be fairlycorrectlrealistic though,

see [ 5j, [6j, although some target types may not be operational today.The radar types associated witheach target type is more uncertain since such data is not readily available.Hencethe ESMpart of thetargettype

identific ation simulations has been selected more to

6

illustrate as many aspects of the functionality as

possible.

The considered scena rio was a South Atlantic one close to the Falkl and s Islands. The target types considered in thescena rioarc listed inTable I.together

with their flightenvclopes as stated in

I

5

I

.

I

6

I

.

Note thatthe intercep tand attack ro/esoftheMirageaircraft

arehandled as different types.The relation between the emi tter mode/radar type measurem ents and the target typesisillustratedin Fig. 2.Aconnectin glinebetween

aradartypeand a targettype meansthatthat parti cular

emirter mode/emitterispossible for thattargettype.See I41for moredetailson how toestimate targettype from emitter mode/rad artypemeasurements.

Theexperimental MST performed real time trackin g

with asynchronousdata from 7 simulatedsensors.There were 4 radars on the Falklands Islands and 3 airborne ESM sensors.The positions of the sensorsarcdisplayed

in Fig.3. The ESM sensors were located at analtitude of 40000 feet. This gives a(40000 feetaltitudeoftarget) aradar horizonofabout 800km. Informationfrom ESM sensormanufacturersindicatethatit isnot unreali sticto

use thisrangeas ESMsensor range against main radar

lobes. so thiswasthe assumption made.The radars werc

assumed to haverangesof250 km.The accuracys(10- )

were 100 m(range).0.10 deg.(azimuth)and0.25deg (elevation). All sensors had scan times of 10S. The radar c1utter was 10-'/km'. foreach radar. The ESM sensor aceurney's were 0.25 deg. (azimuth) and 0.25

deg. (elevation). Such 2-D ESM sensors can be

manufactured today.Note thatthemeasur ements of the ESA! sensorswere subjec t to 30%randomerrors in the simu lations,cf.

I

4

I

.

Only 4 of the possible 9 target types appea red as

targetsin the simulations.Thetarget path sare plottcdin

Fig. 3.The details can besummarizedasfollows

Path l: 5 AVRO Vulcan bombers startingnortheastof the Falklandsat 11000 m,heading 190deg.. speed235

mIs. Turningwest and c1imbingto

I

noo

mover the Falklands. R2 tran snuuingconunuously.Startal time ()

s, target separation 100S.

Path 2:5 AVRO Vuleau bombers starting northcastof the Falklands al 11000 m,heading 190 deg..speed235

mIs. Turningwest and c1imbing10 19200 m north of the Falklands. R2 transmitting continuously. Start at time500 s.largel separat ion 100S.

Path 3: 20 Sea Harriers starting from the Falklands.

heading south west. speed 300 mIs. Steep c1imb 10 14500 m after a few minutes.Maximum c1imbrate 200

mIsfor 20S.R2 transmittinguntil time 1400 s (23min

20 s). then switching10RI. Start altime750 s.target separation60S. P: ol cl cl I' 6: P, H 1 h

(12)

Path 4: 10 Sea Harriers starting from carrier north cast of the Falklands, heading west. speed 250

mis

,

steep c1imb to 14500 m north of the Falklands. Maximum c1imbrate 200

o

m

is

330

m

is

255

m

is

750 km hi-hi-hi

FRS.Mk2

o

m

is

Mirage5P N.A. 650

m

is

685 km 10-10-10

Table I:Flight envelopes for the considered target types.The interceptand attackroles of the Mirageaircraftare handledas different types.

Vulcon Dassault SuperEtendara IsossauttMirage11101.uüercept DussaultMirage5P.uttock SeaHarrierFRS A4k.2 Ale.DonnelAe-Bs" DassuultMirageIJlE4.attack FAMA fA58 Pucara DossaultMirage5P.tntercept

Fig.2:Radaremitter possibilities for each target type.

the n switchingto R7.Start at time 300.target scparatien

60 s.

Path 7:10 A4 Skyhawk starting south of the Falkla nds. Headingtowards the Falklands.Altitude 10000 m,speed 225

m

i s

.

After a while deseending towards 500 m

altitude.R3 transmittingcontinuously. Start time 0 s. target sepa ration 100 s.

Path 8:10 A4 Skyhawk, starting south of the Falklands. Heading north, then towards the Falklands, altitudc 11500 m. speed 225

m

i s

.

After a while deseending

towards 500 m altitude. R3transmitting continuously. Start time 1000 s.target separation120 s.

Examplel: The target paths 1-8 were run with the ESM sensors turned off.The target type identification algorithms therefore only used the experimental f1ight envelopes of the tracks.The algorithms were set up to estimate and to exploit maximum attitude. maximum

(13)

Fig. 3: Target paths and sensor locations.Sensors \-4are radars,while sensors 5-7 are airborne ESM sensors.

flight envelopes of Table I were used in the simulation. In order to obtain a safety margin a linear reduction of the likelihood from \ to 0 was applied outside the Ilight envelope limits, cf. ( 5 ). The widths of the safety margin bands were 500 m for maximum altitude, 10 misfor minimum and maximum speeds and 25misfor maximum climbrate, except for the Sea Harrier and the Mirage 5 that used 75mis.The results appear in Fig. 4, Fig.5, Fig. 6 and Fig.7.In these figures the cumulative target type probabilities of all tracks of each target path are displayed as a function of time. More specifically, the following measure was used

8

for i

=

1,.. . ,9 and tE,. I

=

3,6, ... ,42 minutes. The results are thus two-dimensional plots that describe how the total probability of the tracks in the target path evolve over time.

Example 2: This example corresponds to Example 1 with the exception that the airborne ESM sensors were turned on. As aresuIt, the combination of ESM data and experimental flight envelopes were used. Note again that the ESM data used in the simulalion is heavily corrupted with 30 %random errors.The tuning of the algorithm that uses ESM data was exactly similar to the one described in [ 4

J

.

The results are displayed in Fig.4,

Fig.5, Fig. 6 and Fig.7, using (6).

Discussion: Starting with the Vulcan, path \, in the radar only case, it is clear that the algorithm exploitsthe maximum altitude of the track to exclude first the

(14)

Fig 4:Cumulative target type probabilities ofpath 1 and path 2. IIt S .tal ver :1 ere ,"d ain 'i1y the the .4, the the

(15)

IQ

Fig. 6:Cumulative target type probabilities of path 5 and path 6.

(16)

Pucaraand then the remaining alternatives.In the end of therun the cumulative probability of

1;

is close to 5 which indicates that all targets are correctly classified.

Note the relatively long time that passes until other

alternatives are excluded. When the ESM sensors are used, Fig. 4 shows that exclusion of all alternatives

except

T.,

and

Tz

is al most instantaneous.The reason is

evident from Fig.2.The exclusion of

T.,

is no faster than without ESM sincethe flight envelope needs to be

exploited for thisexclusion.Similar couunents are valid for the second path. However, the procedure is only successful for 4 out of 5 targets in the ESM case. Thus 19out of20 c1assifications are correct.

The Sea Harriers are c1assified using the c1imbrate parameter. as should be evident from the definition of the path s and from astudy of the radar only cases of Fig. 5. Sincethe c1imb comes relativelyearly in both paths .little is gained in terrns of the time to exclusion of otheralternatives than

T.,

and

1;

when ESM sensors are used.lt canbe seen in Fig. 5 that a few errors appear. 55

out of60c1assificationsare correct. lt can also be seen

in Fig. 5that the major souree of error is a remaining probability for Tgand

I;,

.

This is natural,considering therelativelyhigh maximum c1imbrate for these types.

cf. Table I.and the short duration of the c1imb. The c1assification of theMirage aircrafi is done using the maximum speed and the maximum altitude. For path 5 the Mirage 11I and the Mirage 5 cannot be

separated. leaving 4 remaining types at the end of the run,seeFig.6. In path 6 maximum altitude is used to excludealso the Mirage 11I.The major effect ofthe ESM

sensors is that they result in an earlier track initiation.

In the begining ESM data also contribute to the

classification, cf.Fig.2.24 out of 24 classificationsare

correct(correctly ambiguous).

As can be seen from the radar only cases of Fig.7.

littlecan be done to c1assify the A4 Skyhawk with only kinematic infonnation. Only the Pucara can be

excluded.The improvementwhen ESM is added to path

7 isdramatic.both in terrns of time to c1assification and in terrns of c1assification performance, cf. Fig. 2. Unfortunately, there are several errors for path 8.The reasonfor thisappears to bethat thekinematic. passive

on1y estimates become less accurate. This can be

explained bythe late appearance of the targets.which

therefore becomeshadowed byearlier initiated tracks.

As aresuit. the trackscorresponding to path 2 become lessaccurate than the tracks of path I.This is evident from Fig. 7,where the most agile aircraft remain at the

endof therun(path 8).31 out of 40classifications are

correct for theA4 Skyhawkpaths.

6. CONCLUSIONS

The paperdiscussed the identification oftargettypeina

multi-sensor framework.The estimationof targettype

foreach track from experimental flight envelopes was

discussed in detail.Themethod hasbeen implemented

in an experiment al MST system that can also utilize

emitter mode/radar type measurements from ESM sensors. An experimental investigationwas carriedout using a 72 target scenario with 9 target types. The discussed algorithm performed well, with 90 % fu lly correct c1assifications. despite the presence of clutter,

ambiguities and the fact that a large part of thc

c1assification s wereperfonnedwithonlypassive data.

4 J T. Wigren.E.Svics tinsand H.Egnell, "Operational multi-sensor trackinglorair defense", Proc. FirstAustra lia n DataFusio nSymposium.Adelaide,SA.Australia,pp.IJ-I X. Nov.21-22.1996.

( 5

I

D.DonaIdandJ.Lake,Encyclopedia ofWor/dMilitarv

Airc raft, Volume I.Londe n,UK:Airpower Publishing

ua

.

1994.

I

6

I

D.DonaidandJ Lake,Ency clopedia of WorldMilitarv

Airc raft, Volume 2.Londe n,UK:Airpower Publishing

u

a

.

x

..n.

Li,Estima tio nwuiTrackil/g:

Princi p les, Techni ques andSoftware.Norwood.MA:Artech

House,1993.

I

10J E.Svicstins,"Multi-radar tracki ng for theatermissilc defense",Proc.SPIE.SanDiego,Ca,vol. 2561,pp.384-.194,

(17)

[ 11

1

E. Sviestins, "Multi-sensor tracking forair traffic contraIandairdcfense",ArCSystems.vol.I.no.I.pp.1 0-16.1995.

I 12 1 A. P. Dempster. ..A gcneralization of Baycsian inference",J. RoyalStat.Soc..SeriesB,vol. 30,pp. 205-247,1968.

[ 13

I

G. Shafer, A Mathematical Theory of Evidence. Princeton,NJ:PrincetonUnivers ityPress,1976.

(18)

Multi-Sensor Data Fusion

Or. Han

s

Drie

ssen

Holland

se

Si

gnaalapparaten

BV

P.O.

Box 4

2 7550 GD H

engelo

The Netherlands

E

-mail : j

ndriesse n@s ignaa l.nl

Abstract

On

e of

th

e

primary tasks

o

f th

e se

nso r and

c

ammand and

co

ntrol syst

ems

on a military

platf

orm

i

s

t

o

aid in th

e c

ompilation

o

f a r

ecognis ed

pi

cture

of the operational

e

n vironment.

Thi

s

pap

er

pre

sents

r

esults of

th

e

r

esearch

and d

evelopment

a

ctivities c

arried

o

ut

a

t

H

ollandse

Signaalapparat

en

BV for th

e

automation of th

e c

ompilation of this pi

cture

.

Th

e

se

lected multi

-sensor

data fusi

on c

oncept is

a plot l

evel

fusion

sy

stem r

elying o

n a ma

ximum

likelih

oodformulation e

mploying multipl

e

h

yp othesis

tra

cking.

Th

e sys

tem is

co

mposed

of

a

c

orrelation f unctio

n

that pairs lik

ely

plot tra

ck co

mbinations, a maint

e

nance

fu

nct inn

that

p

erform s

data ass

ociation,

tra

ck

initiation

, co

nfi rmation and d

eletion,

a fi

lt

er

function that

e

stimates th

e

tar

get

kin

ematics.

a

c

lassification functi

on

that d

ecides

about th

e

tar

get

cla

ss,

and an

e

valuation fu

n

ction

that allows i

n

corporation o

f prior s

ensor

and

e

n

vironmental

kn

owl edge

.

Imp

ortant fe

atures

of

th

e

MSDF s

ystem

ar

e

(i)

th

e

us

e of

reliable radial sp

eed

m

easurements

through

out

th

e co

mplete pro

cessin g c

hain,

(ii)

th

e

appli

cation

of s

ensor

s

pec ific clutt

er

maps in th

e co

rrelation fun

ction ,

(iii) th

e

f

eedba ck o

f th

e

classification

fun

ction

t

o

th

e f

ilter

w

hich all

ows

th

e a

pplication

of

tar

get

jli

ght e

nvelopes, and

(

iv) th

e

incorporation of

s

ensor and

e

nvironmental knowl

edge

.

Based

011th

e

evaluation

o

f

t

he syst

em

o

n a radar and infrar

ed

s

ensor

suit

e.

it

c

an b

een c

oncluded that th

e

s

elected co

ncept and

al

gorithms

not

o

nly mak

e

th

e

fusionfea

sible,

but ar

e

hi

ghly

jl

exible

with r

espect

10

the s

ensor

data

c

haracteristics, and will

e

ven gr

eat

l

y

impro

ve

th

e

p

e

rforman

ce

of a

m

ono-sensor

sys

tem

.

1. Intro

d uetion

One of the primary ta

sks

of the sen

sor

and command and control

s

yste ms on a military

I)

platform i

s

to aid in the compi

lation

of a recognised picture of the operational environment.

During a co

urse

of action amission needs to be fu

lfilled

,

and t

he

recognise

d

picture serve

s

to plan future actions in order to optimise the

s

uccessful completion of the mission

.

A

s

et of

ba

sic

question

s

needs t

o

be answered within thi

s

context. First of all one

i

s

intere

sted

in the

qu

estions

whether there are any targets in the environment

,

and if so w

here

they are a

nd

where they are going to

.

Second

l

y,

one would like to know whether these target

s

are friend

ly,

n

eutra

l

or hostile

,

a

nd

what types of targets they are

.

Fina

lly,

the question needs to be

an

swered

whether or not these target

s

are a threat for the

s

uccessfu

l

co

mplet

ion of t

he

mi

ssion .

That part of t

he

se

nsor

suite on a platfo

rm

that aids in the compila

tion

of a

r

ecognised

1) Although the techniquespresenredin thispaper areapplicable in a more general cont ext,thispaper focuses on militaryapplications.Moreover,thepresenlatio nconcentrates on navy platforms.

(19)

picture of the environment commonly comprises one ore more surveillance radars, a

navigation radar, an infrared search and track sensor (IRST), an identification friend or foe

(lFF) sensor, and an electronic support measures (ESM) system

.

Such a suite represents a set

of more or less dissimilar sensors, each sensor measuring an amount of information that only

partially overlaps with information from other sensors. Additional information sourees are

data links that allow for the reception of a part of the recognised picture of other

accompanying platforms, geographical positioning systems (GPS), which data may be

distributed on the data link as weil, and digital maps of the environment as available in

geographic information systems (GIS).

This paper focuses on the results from the research and development activities in the field

mentioned above carried out at Hollandse Signaalapparaten, Signaal for short. The objective

of these data fusion activities is

(i)

to automate the above picture compilation process in order

to relax the operator workload, and (ii) to additionally improve the picture quality. The aim

is to develop a generic multi-sensor data fusion system (MSDF) that properly fuses the

available sensor data into a recognised picture exploiting the available prior knowIedge,

while it is independent as much as possible of the specific sensor suite and knowiedge.

2. The Multi-Sensor Data Fusion Concept

In this section, the MSDF concept is described. Among the first choices to be made in a

sensor data fusion system are the level at which the fusion is performed, as weil as the

mathematical basis the fusion relies upon, and which determines the decomposition into

building blocks and their interfaces.

2.1 Operational Benefits from Plot Level Fusion

The selection of the basic level of fusion effectively comes down to the choice between a plot

or track level fusion system, since a signal level fusion seems to be far beyond practical

realisation. Apart from the issue of redundancy andJor graceful degradation, the choice

mentioned above is equivalent with the choice whether or not feedback should be applied

within a hierarchical decentralised fusion system. A track level fusion system is a prototype

of a hierarchical decentralised fusion system

without feedback, while a plot level fusion

system is the centralised counterpart of a hierarchical decentralised fusion system

with

feedback.

Plot level fusion systems that receive all plots of the sensors in the suite have a number

of important operational benefits over track level fusion systems. This can theoretically be

explained by the fact that in specific situations in a track level fusion system information is

being lost or is hindered by latency.

The first operational benefit that plot level fusion systems offer is the improvement in

reaction time. When two sensor systems initiate a track based on a two out of three plot

detection criterion, a track level fusion system is as fast as the fastest sensor. On the other

hand, when both sensors supply their raw plot data to a plot level fusion system, the system

is at least as fast as the fastest sensor, usually faster.

An additional operational benefit is improved track continuity and track accuracy

during manoeuvres especially in c1uttered regions. Again consider two sensors observing a

manoeuvring target in cutter, then if one of the sensors looses track of the target, the

16

(20)

performance of a track level fusion system immediately becomes worse, due to the lack of

infonnation. In case the second sensor also looses track on the target, the system track is lost

as weil, provided of course the first sensor is not able to initiate a new track on the same

target.

It

will be clear that the plot level fusion system receives all data and has an improved

track accuracy if one of the sensor tracks is lost, and a higher chance of continuing the track

if both sensor tracks are lost.

Another operational benefit is improved track continuity and track accuracy in case of

crossing or formation f1ying targets. Again, due to the loss of information in case of a track

level fusion system when one or more sensors looses track of one of the targets, the plot level

fusion system will outperform a track level fusion system

.

The next operational benefit is the cueing of sensors that have different coverage.

Consider an incoming target observed by a medium range radar and by a long range radar.

Then in a track level fusion system, areliabie decision to fuse both sensor tracks cannot be

made until the track of the medium range radar is reported to the fusion system and the track

is run-in. In a plot level fusion system, a stabIe system track is reached earl ier.

The final operational benefit is improved track continuity and track accuracy during

emission control (EMCON) scenarios, where restricted use is being made of active sensors,

or in ajamming environment. In both cases, specific individual plots may be lost, which may

lead to sensor track losses. The plot level fusion system will generally be better suited to

continue the system tracks and achieve an improved accuracy compared with the track level

fusion system.

For these reasons, Signaal has selected the plot level fusion system as the basis of their

MSDF system. At the same time this choice does not restriet the fusion of sensors that only

provide track reports, since temporal decorrelation techniques can be applied to these reports

such that their further processing can be performed by the plot level fusion system.

2.2 Outline of the System Concept

The MSDF concept relies on maximum likelihood (ML) techniques that try to explain the

incoming plot and track data by the most likely configuration of tracks and false alarms. To

that end, the multiple hypothesis tracker (MHT) concept has been selected as the most mature

ML realization, which computational demand can be met by current hardware.

As a result of this choice, the MSDF concept can be decomposed into several building

blocks. In Figure I

.,

the building blocks of the MSDF system concept are shown

.

The system

can be subdivided into a number of functions:

-

The

correlators

find all plots belonging to a certain track. A single

correlator

therefore

perfonns exactly as a correlator in a conventional system. Due to sensor specific aspects

involved in the correlation process a

correlation

function is needed for each sensor from

which data is fused.

- The

overlap

function perfonns the time management of the different sensors. For

example one task of

overlap

is to solve the problems occurring when a quickly rotating

sensor overtakes a slowly rotating sensor. Since the

overlap

function is a system

implementation oriented function it will not be further discussed in this summary.

- The

maintenance

function performs the track maintenance, i.e. the initiation and

deletion of tracks. Due to the choice of the MHT method the function inherently also

perfonns the data association function.

(21)

matrix

(

i.e

.

track accuracy

)

of a track

.

Also thi

s

function account

s

for e

stimating

s

ystematic sensor error

s

and

s

ensor parallax

.

-

The classification function use

s

plot and track atlribute

s

to det

ermine

the tar

get' s

c1

ass

(

e.g. jet

,

propeller

,

helicopter

,

non

-target)

-

Evaluate performs processing of the track

s

using

s

pecific prior

s

ensor and

environmental knowiedge

.

Note that no counterpart for

ev

aluate i

s

pre

sent i

n

conventio

nal

tracking systems.

Figure 1. Layout of the MSDF system

Sensor r---, Da ta ---.. Corre la tor Senso r r--~~--' Data - . .Correlator Filte r

Overlap 1---1~ Maintai n Classify Evaluate Tracks

3 .

De

scription

of th

e B

uilding Bl

ocks

3.1 Th

e C

or r ela tion

F

unction

T

he

p

urpose

of t

he

corre

lation

fu

nction

is to es

tablish

relations between

n

ew sensor

o

bservations

(p

lots)

a

nd

target

t

racks. All

pl

ausible p

lot

track com

binatio ns

are fo

und

by th

e

correla

tors

a

nd

a corre

lation

score (as req

uired in

M

HT)

is comp

uted

for t

hese

combi

nation

s

.

Th

e c

orrelator

rece

ives th

e

t

rack state vec

tor

a

nd

covaria

nce

ma

trix

calc

ulated

by

t

he

f

ilter.

A

t

firs

t, th

e

tr

ack s

tate

vec

tor

a

nd

covaria

nce m

atrix are predicted to the time

t

he sen

sor

is expec

ted t

o

d

etect t

he t

arget. T

his

res

ults

i

n

t

he

p

redicted

position of the target and it

s

p

rediction

accu

racy

.

Next, t

he

predic

ted

sta

te

vec

tor

a

nd

covariance matrix are transformed

to the loca

l

senso

r

coo

rdinates

in order to form a predicted mea

surement

and associated

covariance

m

atrix, thereby compensating for sensor parallax

.

Then the sensor measurem

ent

is corrected for the estimated systematic errors

,

and the measurement inaccuracies and

variances of the estimated sy

stematic

errors are added to the covariance matrix

.

Finall

y,

th

e

correlation score is calculated ba

sed

on the difference between the actual and the pred

icted

measurement.

A correlator is present for each sensor. Basically the correlation function is a

s

tandard

process which is influenced by the typical sensor characteristic

s.

The correlator

s

are

the

refore

identical but controlled by the specific sensor knowIedge. Among the st

andard

(22)

aspects of a sensor are: the sensor coverage area, the revolution time, the entities of the

observations, the standard deviations of each measurement, the probability of detection

,

and

the false alarm probability

.

A number of Signaal radars provide target range

,

azimuth, elevation and radial speed.

However, since the range and radia

l

speed can sometimes be ambiguous, special processing

is needed in this case.This processing is based on fitting all fold possibilities

.

IRST's

generally provide very accurate target azimuth and elevation. This implies that correlation

can and will only be done with these two entities

.

The generic nature of the correlation

function ensures that the correlation scores of radars and IRST's is comparable

.

For each sensor a c1utter map can be maintained in the correlation process. This enables

the determination of alocal false a

larm

probabi

lity

indicating c1utter areas

.

This false alarm

probability is used in the maintenance function in combination with the track correlation

s

core and the probability of detection in order to determine the plot track association and the

initiation of new tracks.

3.2 The Maintenance Function

Conventionally the track maintenance function takes care of the track life cycle

,

which

comprises track initiation, track confirmation, and track lo

st

declaration.

In

the presented

MSDF concept the maintenance function also includes data association

,

which is deciding

about the actual plot track assignments. This is primarily due to the app

lication

of MHT

,

which offers a generali

sation

of both traditional maintenance and association activities

.

The principle of MHT is

10

delay difficult assignment decisions until more data

becomes available by recursively building a tree of likely assignment hypotheses and by

a

ssociating

a posterior probability to each hypothesis. This can improve the system

performance such as in the case of formations and crossing targets

.

The maintenance function receives for each plot all possible track assignments from the

correlators, and it then creates the following hypotheses

:

the plot i

s

the start of a new track

,

the plot is a false alarm

,

and finally for each possible plot track combination, the plot belongs

to the track

.

In

this way a hypothesis tree is created for a set of plots from a sensor

,

where each

branch of the tree represents an assignment solut

ion

for all plots of the set. After this the next

set can be processed

,

which can be data from an other

s

ensor or data from the same sensor

(the next scan)

.

For each hypothesis the probability is calculated. The tracks of all hypotheses

that are considered are sent to the filter.

The hypothesis tree grows exponentially and therefore

,

measures must be taken in order

to reduce this computational and memory load

.

The first measure i

s

to di vide the hypothesis

tree into smaller independent hypothesis trees

.

This procedure is referred to as clustering and

does not introduce any artifacts

.

The second measure is to prune the hypothesis tree

.

Basically three forms of pruning are applied

:

thresholding

,

i

.e.

deleting hypothe

sis

with a

probability lower than a certain threshold, breadth control, i

.e.

allowing only a fixed number

of hypotheses (with the highest probabilities) at a certain time

,

and depth control, i.e

.

not

allowing decisions to be delayed longer than a certain time interval.

Track initiation is performed by creating the 'new track' hypothesis

.

A track becomes

confirmed when the

'

new track' hypothesis is accepted, i.e. when all other possibilities have

been pruned. A track is declared lost when for a certain time period no observations are

assigned to the track

.

This time period can be default or be controlled by the evaluate function

(e

.g.

for tangenrial fading prediction)

.

(23)

3.3 The Filtering Function

Each assignment that the maintenance function investigate

s

must be filtered. The filter mu

st

generate an estimate of the track state vector and a covariance matrix, and also estimate the

sensor parallax and systematic errors. Furthermore, due to the fusion performed the filter h

as

some additional constraints

.

Observations are received at irregular time intervals and the

filter should be able to process them appropriately. Observations contain different

measurement en tities (e.g., infrared or ESM observations contain neither range nor radial

speed). Observations have different accuracies; azimuth and elevation measurements from

infrared systems are far more accurate than from a surveillance radar.

In conventional radar systems often a form of

a~