Longitudinal diffusion MRI analysis using Segis-Net
A single-step deep-learning framework for simultaneous segmentation and registration
Li, Bo; Niessen, Wiro J.; Klein, Stefan; de Groot, Marius; Ikram, M. Arfan; Vernooij, Meike W.; Bron, Esther
E.
DOI
10.1016/j.neuroimage.2021.118004
Publication date
2021
Document Version
Final published version
Published in
NeuroImage
Citation (APA)
Li, B., Niessen, W. J., Klein, S., de Groot, M., Ikram, M. A., Vernooij, M. W., & Bron, E. E. (2021).
Longitudinal diffusion MRI analysis using Segis-Net: A single-step deep-learning framework for
simultaneous segmentation and registration. NeuroImage, 235, [118004].
https://doi.org/10.1016/j.neuroimage.2021.118004
Important note
To cite this publication, please use the final published version (if applicable).
Please check the document version above.
Copyright
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons. Takedown policy
Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim.
This work is downloaded from Delft University of Technology.
ContentslistsavailableatScienceDirect
NeuroImage
journalhomepage:www.elsevier.com/locate/neuroimage
Longitudinal
diffusion
MRI
analysis
using
Segis-Net:
A
single-step
deep-learning
framework
for
simultaneous
segmentation
and
registration
☆
Bo
Li
a,∗,
Wiro
J.
Niessen
a,b,
Stefan
Klein
a,
Marius
de
Groot
a,c,
M.
Arfan
Ikram
a,c,d,
Meike
W.
Vernooij
a,c,
Esther
E.
Bron
aa Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, the Netherlands b Imaging Physics, Applied Sciences, Delft University of Technology, the Netherlands c Department of Epidemiology, Erasmus MC, Rotterdam, the Netherlands d Department of Neurology, Erasmus MC, Rotterdam, the Netherlands
a
r
t
i
c
l
e
i
n
f
o
Keywords: Segmentation Registration Diffusion MRI Deep learning CNN Longitudinal White matter tracta
b
s
t
r
a
c
t
Thisworkpresentsasingle-stepdeep-learningframeworkforlongitudinalimageanalysis,coinedSegis-Net.To optimallyexploitinformationavailableinlongitudinaldata,thismethodconcurrentlylearnsamulti-class seg-mentationandnonlinearregistration.Segmentationandregistrationaremodeledusingaconvolutionalneural networkandoptimizedsimultaneouslyfortheirmutualbenefit.Anobjectivefunctionthatoptimizesspatial correspondenceforthesegmentedstructuresacrosstime-pointsisproposed.WeappliedSegis-Nettothe anal-ysisofwhitemattertractsfromN=8045longitudinalbrainMRIdatasetsof3249elderlyindividuals.Segis-Net approachshowedasignificantincreaseinregistrationaccuracy,spatio-temporalsegmentationconsistency,and reproducibilitycomparedwithtwomultistagepipelines.Thisalsoledtoasignificantreductioninthesample-size thatwouldberequiredtoachievethesamestatisticalpowerinanalyzingtract-specificmeasures.Thus,weexpect thatSegis-Netcanserveasanewreliabletooltosupportlongitudinalimagingstudiestoinvestigatemacro-and microstructuralbrainchangesovertime.
1. Introduction
Theincreasingavailabilityoflongitudinalimagingdataisexpanding ourabilitytocaptureandcharacterizeprogressiveanatomicalchanges, rangingfromnormalchangesinthelifespan,toresponsesalongdisease trajectoriesortherapeuticactions.Comparedtocross-sectionalstudies, longitudinalimagingstudieshavetheadvantageofallowingtotracethe orderofeventsattheindividuallevelandtocorrectforthe confound-ingeffectoftime-invariantindividualdifferences(vanderKriekeetal., 2017).Theyarethusconsideredtobemoreaccurateandsensitivein capturingsubtlechangesovertime.Toanalyzespatio-temporalchanges fromlongitudinalimagingdata,atailoredframeworkthatinvolvesboth segmentationandregistrationisrequiredtosegmentthe structures-of-interestandtoregistertemporalframes.Thiscanbeachievedbydirectly combiningtwoexistingsegmentationandregistrationtools,whichare oftendesignedforcross-sectionalstudies.However,theinformation of-feredinlongitudinaldataremainsunderutilized.
Variousstudieshaveshownthatcombiningsegmentationand reg-istrationatthestageofalgorithmoptimizationcanleadtoimproved
Abbreviations:MRI,MagneticResonanceImaging;DTI,DiffusionTensorImaging;FA,FractionalAnisotropy;MD,MeanDiffusivity;TE,EchoTime;TR,Repetition Time.
☆SpecialIssueonLongitudinalNeuroimaging. ∗Correspondingauthor.
E-mailaddress:b.li@erasmusmc.nl(B.Li).
performance. Apopularcombinationstrategyis tousetheoutputof onetasktooptimizetheother.Registrationcanbeimprovedbyusing segmentation-levelcorrespondencesasinputfordeformation initializa-tion(DaiandKhorram,1999;Postelnicuetal.,2008)andoptimization (Balakrishnanetal., 2019;Bastiaansen etal., 2020;De Grootetal., 2013b;Huetal.,2018;Rohé etal.,2017;Zhuetal.,2020).Likewise, segmentationcanbenefitfromregistrationbypropagatinganatomical informationtosubsequentframes,ashasbeenshowninclassical multi-atlasbasedsegmentationmethods(Fischletal.,2002; Vakalopoulou etal.,2018)andinrecentdata-augmentationtechniqueswhich intro-ducelabelstosupportunsupervised(Pathaketal.,2017)and weakly-supervisedsegmentation(Bortsovaetal.,2019;Vlontzosand Mikola-jczyk,2018).
Otherapproachescombinetheoptimizationofparametersfromboth tasks on a deeperlevel. WyattandNoble (2003) sub-groupedthese methods intotwo typesaccordingtothe wayin which theyupdate theirparameters:(1)“simultaneousestimation” thatupdatesboththe classlabelsandthetransformationsinasingle-stepoptimization,and (2)“jointestimation” that alternatelyupdates(separate)modelsina multi-step optimization.Althoughtheinitializationandrobustnessof
https://doi.org/10.1016/j.neuroimage.2021.118004
Received20December2020;Receivedinrevisedform12March2021;Accepted19March2021 Availableonline29March2021
1053-8119/© 2021TheAuthor(s).PublishedbyElsevierInc.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense (http://creativecommons.org/licenses/by-nc-nd/4.0/)
jointestimationcanbeinfluencedbytheselectionoftheorderto op-timizeandthecriteriatoswitchtasks,thisapproachispreferredasit requireslesscomputationpowerandallowstousetask-specifictraining datasets(AshburnerandFriston,2005;Chengetal.,2017;Gooyaetal., 2011;Parisotetal.,2014;Pohletal.,2006;WyattandNoble,2003;Xu andNiethammer,2019;Yezzietal.,2003).Simultaneousestimationis expectedtobemoreaccurate,asitfullyexploitstheconditional correla-tionsbetweentwotasksthatcanbediscountedinsequentialprocessing (AshburnerandFriston,2005).Inaddition,simultaneousestimationcan explicitlyoptimizeperformancesthatrelyonbothtasks.Weexpectthat thisadvantagehasalargepotentialinimprovingthereliabilityof anal-ysisoflongitudinalimagingdata,forinstancebyoptimizingthe spatio-temporalconsistencyofthesegmentation.Withthegrowing capabil-ityofmodelingandcomputationbydeeplearningtechniques,several simultaneousmethodshavebeenproposedandcoupledsegmentation withdeformableregistrationindifferentways,eitherfor2D(Qinetal., 2018)or3Dimages(Estienneetal.,2020;2019;Lietal.,2019).
Diffusionmagneticresonanceimaging(MRI)isanon-invasive imag-ingtechniquethatmeasuresthediffusionofwaterin-vivoandcanbe usedtoquantitativelycharacterizewhitematter(WM)microstructure. Inaddition,diffusionMRIderivedmeasures,suchasdiffusiontensor imaging(DTI)metrics(LeBihanetal.,2001),arelikelytobemore sen-sitivethanstructuralmeasuresintheearlydetectionofchangesinWM, andarethereforepromisingfortheidentificationofsubtlechangesthat relatetotheearlystagesofthedisease(Niessen,2016),forinstancein studyingdementiasubtypes(Meijboometal.,2019).Longitudinal diffu-sionMRIhasbeenwidelystudiedatvariouslevels,i.e.,from regions-of-interest(Keihaninejadetal.,2013;Sullivanetal.,2010),totractlevel (Dimondetal.,2020;LebelandBeaulieu,2011;Meijboometal.,2019; Yendikietal.,2016),andvoxellevel(Barricketal.,2010;Farbotaetal., 2012;DeGrootetal.,2016).SinceWMtractsarefunctionallygrouped axonalfibersandthoughttosubserveparticularbrainfunctions, tract-specificinvestigation mayhighlightcategoricaldifferences in vulner-abilitytoneurodegenerationandbridgetheinterpretationofimaging biomarkerswithclinicalsymptoms.
SegmentationofWMtractsishowevernon-trivialbecausetracts can-notbeidentifieddirectlyfromdiffusionMRI,i.e.,there isnoin-vivo “gold standard” fortract (CrickandJones,1993),andbecausetheir anatomycanbecomplex.WMtractsarecommonlysegmentedbased ondiffusiontractographybyreconstructionofpotentialfiberpathways (Conturoetal.,1999).Recently,deeplearningbasedmethods,in par-ticularusingconvolutionalneuralnetworks(CNN),haveemergedand showedpromisingaccuracyandefficiencyinsegmentingWMtracts(Li etal.,2020a;2018;Wasserthaletal.,2018).
InthepresentworkwefocusonaCNN-basedframeworkfor longi-tudinalanalysisofWMtracts,i.e.,Segis-Net,andinvestigatethevalue ofsimultaneousoptimizationofsegmentationandregistrationinthis setting.In(Lietal.,2019),weintroducedagenericframeworkfor si-multaneousoptimization,inwhichincreasedaccuraciesofbothtasks wereobservedinapilotanalysisofasingletract(forcepsminor;FMI). Inthispaper,weextendthetract-specificmethodbyenabling concur-rentsegmentationof multipletracts,which isanon-trivialtask asa voxelcanbelongtomultipletracts.Thisalsosolvestheproblemof in-consistenciesin deformationsbecauseoftract-specificROIs.The reg-istrationtaskwithintheframeworkisupdatedtolearnonlylocal de-formationsratherthananend-to-endcompositeincludingrigid trans-formation,asbrainlocalchangesovertimeisafocusin longitudinal imagingstudies.Inaddition,wecomparetheperformanceofSegis-Net totwomultistagepipelinesbasedonbothclassicalanddeeplearning algorithms,andtwostate-of-the-artmethods.Thesegmentation accu-racy,registration accuracy,spatio-temporalconsistencyof segmenta-tion,andreproducibilityofsegmentationandtract-specificmeasuresof thepipelinesarequantitativelyevaluated.Also,weevaluatethe sample-sizereductionthatcanbeachievedintheimaginganalysisofWMtracts toprovideinsightinto thepractical valueof themethodsin clinical applications.
2. Methods
Inthissection,wefirstdescribehowthesegmentationand registra-tiontasksareindividuallymodeledusingCNN-basedapproaches. Sub-sequently,wepresenttheproposedSegis-Netthatintegratesbothtasks inasingle-stepCNNframework.
2.1. CNN-basedimagesegmentation
Givena𝑛-Dimage𝐼 whichcanbedescribedbyeitherintensity val-ues,multi-channelfeaturesordirectionaltensors,thegoalofCNN-based segmentationistoautomaticallyinfer,foreachvoxel𝑥∈ℝ𝑛,its prob-abilityofbelonging tothestructure𝑘∈ [1,𝐾]with𝐾 thenumberof structures,i.e.,voxel-wiseclassification.TheCNNmodelcanbe inter-pretedasaparameterizedmappingfunction𝚯suchthatthesegmented structuresspatiallycorrespondtoasegmentationgroundtruthwith mul-tiplechannels={𝑆1,…,𝑆𝐾}.Theestimationofthesegmentationis
formulatedas:
̂=𝚯(𝐼). (1)
𝚯 is commonly modeled by a nested series of convolutions, non-linearity,normalization,andre-samplingoperationsembeddedinthe networkarchitecture.𝚯 indicatetrainableparameters.
Theprocedureofestimatingparameter𝚯 isthendefinedasan opti-mizationwithrespecttoalossfunction𝑠𝑒𝑔,aimingatminimizingthe
classificationerroroverallthe𝑁 pairsoftrainingsamples{(𝑖,𝐼𝑖)}𝑁𝑖=1, i.e., 𝚯 ←argmin 𝚯 𝑁 ∑ 𝑖=1 𝑠𝑒𝑔 ( 𝑖, 𝚯(𝐼𝑖) ) . (2)
Thelossfunctioncomprisesmetricsthatquantifythedifference be-tweenthepredictionandthegroundtruth.Inthisstudy,𝑠𝑒𝑔isthe
aver-ageDicecoefficient(Crumetal.,2006;Dice,1945)overall𝐾 structures:
𝑠𝑒𝑔(, ̂)=−𝐾2 𝐾 ∑ 𝑘=1 ∑ 𝑥𝑆𝑘̂𝑆𝑘 ∑ 𝑥(𝑆𝑘)2+∑𝑥(𝑆̂𝑘)2 . (3)
Afterestimationofthemapfunction𝚯,aprobabilisticprediction forthestructuresofinterest ̂inagivenimagecanbeinferred(Eq.1).
2.2. CNN-baseddeformableregistration
Letusconsiderapairof𝑛-Dimages,𝐼𝑠inthesourcespaceΩ𝑠⊂ ℝ𝑛,
and𝐼𝑡inthetargetspaceΩ𝑡⊂ ℝ𝑛,whichcontainacommonstructure tobealigned.Thespatialcorrespondencebetweenimagescanbe estab-lishedbyestimatingadensedisplacementfield𝝓,suchthat𝐼𝑠◦𝝓 and𝐼𝑡
correspondspatially.
Inlinewiththehierarchicaloptimizationschemeofclassical reg-istrationalgorithms,mostexistinglearning-basedregistrationmethods useaffinealignmentasaprepossessingstep,inwhichcasethe displace-mentfielddenotesthecompositionofaffineanddeformabletransform, i.e.,𝝓 =𝝓𝐴◦𝝓𝐷.Toestimate𝝓𝐷,theCNNmodelcanbeinterpretedas
ashareddomain-invariantmappingfunction𝚿suchthatforany un-seenpairofimagesamostlikelytransformationbetweenthemcanbe inferredwithoutpair-specificoptimization,i.e.,
̂𝝓𝐷=𝚿(𝐼𝑡,𝐼𝑠◦𝝓𝐴) (4)
⇒ 𝐼𝑠◦ ̂𝝓 ←𝚿(𝐼𝑡,𝐼𝑠,𝝓𝐴). (5)
Theparameters𝚿 ofthemappingfunctionareoptimizedbasedon a registration dissimilarityloss 𝑟𝑒𝑔, aimed at minimizing the regis-trationerror.Meanwhile,topenalizelargedeviationsofdeformation andpreserveanatomicaltopologyduringtransformations,a deforma-tionsmoothnessterm𝑑𝑒𝑓 iscommonlyincludedinthelossfunction. Inthiswork,weusethemeansquarederrorbasedonintensitiesfor𝑟𝑒𝑔
Fig.1.OverviewoftheSegis-Netframework.𝚯 and𝚿 denotetheparametersofthesegmentation(𝚯)andregistration(𝚿)function,respectively.Blackcircle
indicatesspatialwarpwithaffinematrix(𝝓𝐴)orthecompositedisplacementfield(̂𝝓).Theconcatenationoftheaffine-alignedimagesisusedastheinputfor𝚿.
Lossfunctionconsistsof𝑠𝑒𝑔,𝑐𝑜𝑚,𝑟𝑒𝑔and𝑑𝑒𝑓terms.Solidlinesindicatetheprimaryworkflowofthemethod;dashedlinesindicatetheoperationsthatareonly
implementedduringtrainingandcouldbeadaptedforapplications.
andtheaveragespatialgradientsofthedisplacementfieldfor𝑑𝑒𝑓,i.e.,
𝑟𝑒𝑔(𝐼𝑡,𝐼𝑠◦ ̂𝝓)=|Ω1 𝑡|‖𝐼𝑡−𝐼𝑠◦ ̂𝝓‖ 2 2, (6) 𝑑𝑒𝑓(̂𝝓𝐷)=|Ω1 𝑡|‖∇̂𝝓𝐷‖ 2 2. (7)
CombiningEqs. (4),(5),(6)and(7),theestimationof𝚿 overallthe𝑁
trainingsamples{(𝐼𝑡𝑖,𝐼𝑠𝑖,𝝓𝑖𝐴)}𝑁𝑖=1canbeformulatedas:
𝚿 ←argmin 𝚿 𝑁 ∑ 𝑖=1 𝑟𝑒𝑔(𝐼𝑡𝑖,𝚿(𝐼𝑡𝑖,𝐼𝑠𝑖,𝝓𝑖𝐴))+𝑑𝑒𝑓(𝚿(𝐼𝑡𝑖,𝐼𝑠𝑖◦𝝓𝑖𝐴)). (8)
2.3. Simultaneousestimationofsegmentationandregistration
Inthiswork,weaimtosimultaneouslyestimatetheparametersfor segmentation(𝚯)andforregistration(𝚿)inasingle-stepoptimization. Forthispurpose,weintegratethesegmentationandregistration func-tion𝚯 and𝚿 usinganend-to-endoptimizationwiththeSegis-Net. ThelossfunctionoftheSegis-Netisdesignedtomeetthejointobjective ofboth tasksandmeanwhiletooptimizethespatio-temporal consis-tencyofsegmentationwhichrelyonbothtasks.Theoverviewofthe proposedframeworkisillustratedinFig.1.Wedescribetheframework architectureandlossfunctioninthefollowingparagraphs.
2.3.1. Segis-Netframework
Inthepresentstudy,wefocusontheanalysisof3Dimagesandutilize 3DconvolutionsfortheSegis-Netframework.Theframeworkinvolves function𝚯and𝚽astwoparallelstreamsthatinteractontheir out-puts.Inordertoeliminatethelossinimagequalitycausedbymultiple interpolations,Segis-Netwarpssourceimageswithonlythe compos-itedisplacementfields(𝝓)bytakingasinputtheoriginalsourceimage (𝐼𝑠)andpre-estimatedaffinematrix(𝝓𝐴).Thisdesignhasadditional ad-vantagesoverexistingmethodsthatprepareallorderedpairsof affine-alignedimagesindiskstorage,asonlyuptohalfthestorageisneeded andasitcanbeflexiblyappliedtorelatedimagesinthesamespacesuch astheDTImetrics.
𝚯outputsasetofprobabilisticsegmentations(̂𝑠)ofthesource
image.𝚿outputsadenselocaldisplacement ̂𝝓𝐷alongthex,y,andz axes.Thesourceimageanditssegmentationsaresubsequentlywarped
intothetargetspaceusingthecomposeddisplacementfield.Thewarp operationisimplementedbyacomputationallayerwithdifferentiable trilinearinterpolation(Balakrishnanetal.,2019;Jaderbergetal.,2015). Thesegmentationandregistrationstreamshaveindependentnetwork architectures whichareonlyconnectedbytheoutput,i.e.,the trans-formedsource-segmentationtothetargetspace(̂𝑠◦ ̂𝝓).Thus,theycan beappliedseparatelyaftertakingadvantageofthesimultaneous opti-mization.TheSegis-Netframeworkgivesfouroutputsduringtraining:
1. Thesegmentationofthestructuresofinterestfromthesourceimage (̂𝑠),
2. A localdisplacementfieldbetween the sourceandtarget images (̂𝝓𝐷),
3. Thewarpedsourceimageinthetargetspace(𝐼𝑠◦ ̂𝝓),
4. Thewarpedsourcesegmentationsinthetargetspace(̂𝑠◦ ̂𝝓). We proposea genericframeworkwherethe architectureof each streamcanbe adaptedbasedonspecificapplications. Forthe partic-ularnetworkusedinthisstudy,weencodedtwostreamswithaU-Net architecture,thatwasmodifiedasdetailedbelow(Ronnebergeretal., 2015).Inshort,eachstreamwascomposedofanencoderanddecoder pathwithskipconnectionsoffeaturepyramidatmultiplescalesin or-dertomergecoarse-andfine-convolvedfeatures,similartothe multi-resolutionstrategyusedinclassicalalgorithmstoincreaserobustness. The encoderpaths with max-poolingoperationbetween convolution layersgraduallyextractabstractfeaturesforthetargetanatomy(𝚯) andglobaltransformationbetweenimages(𝚿).Subsequently,the de-coderpathsrestorethedetailsinsegmentations(𝚯)andrefinelocal deformations(𝚿)bylinearup-samplingthefeaturemapsand concate-natingthemwiththecoarsecounterpartatthesamescale.The convo-lutionlayersproduceasetoffeaturemapsbyindividuallyconvolving inputswith3Dkernelsofsize(3,3,3),followedbybatchnormalization (IoffeandSzegedy,2015)andaleakyReLulayer(𝑎=0.2)formodeling non-linearity(Maasetal.,2013).Forthesegmentationstream(𝚯),we splittheoutputlayerintosub-branchestofacilitatemulti-class classifi-cationforvoxelswithmultiplelabels.Thefinallayerofthesub-branches consistedofa(1,1,1)convolutionandasigmoidactivation.Forthe reg-istrationstream,theoutputlayerwasaconvolutionlayerwiththree kernelsthatyieldedthelocaldisplacement̂𝝓𝐷.Weprovidedetailed
im-plementationofthenetworkarchitectureinthesupplementarymaterial (Figs.8and9).
2.3.2. Segis-Netlossfunction
ThelossfunctionofSegis-Netiscomposedoffourtermsthat mea-suresegmentationaccuracy(𝑠𝑒𝑔,Eq.(3)),intensitysimilaritybetween
registeredimages(𝑟𝑒𝑔,Eq.(6)),deformationfieldsmoothness(𝑑𝑒𝑓,
Eq.(7)),andlongitudinalcompositeofregistrationandsegmentation (𝑐𝑜𝑚,Eq.(11)).Itisformulatedas:
=𝑠𝑒𝑔(𝑠, ̂𝑠)
+𝛼𝑟𝑒𝑔(𝐼𝑡,𝐼𝑠◦ ̂𝝓))+𝛽𝑑𝑒𝑓( ̂𝝓𝐷) +𝛾𝑐𝑜𝑚(𝑡, ̂𝑠◦ ̂𝝓),
(9)
and optimized for 𝚯 and 𝚿 over all 𝑁 training samples {(𝑡𝑖,𝑠𝑖,𝐼𝑡𝑖,𝐼𝑠𝑖,𝝓𝑖𝐴)}𝑁𝑖=1:
𝚯,𝚿 ←argmin𝚯, 𝚿∑𝑁𝑖=1𝑠𝑒𝑔(𝑠𝑖,𝚯(𝐼𝑠𝑖))
+𝛼𝑟𝑒𝑔(𝐼𝑡𝑖,𝚿(𝐼𝑡𝑖,𝐼𝑠𝑖,𝝓𝐴𝑖))+𝛽𝑑𝑒𝑓(𝚿(𝐼𝑡𝑖,𝐼𝑠𝑖◦𝝓𝑖𝐴)) +𝛾𝑐𝑜𝑚(𝑡𝑖,𝚯(𝐼𝑠𝑖),𝚿(𝐼𝑡𝑖,𝐼𝑠𝑖,𝝓𝑖𝐴)).
(10)
Wequantifythelongitudinalcompositelosstermusingtheaverage Dicecoefficientoverall𝐾 structures:
𝑐𝑜𝑚(𝑡, ̂𝑠◦ ̂𝝓)=−𝐾2 𝐾 ∑ 𝑘=1 ∑ 𝑥∈Ω𝑡𝑆𝑡𝑘(̂𝑆𝑠𝑘◦ ̂𝝓) ∑ 𝑥∈Ω𝑡(𝑆𝑡𝑘)2+ ∑ 𝑥∈Ω𝑡(̂𝑆𝑘𝑠◦ ̂𝝓)2 . (11)
Inlongitudinalimagingstudies,thespatialcorrespondencein segmenta-tiondependsontheperformanceofboththesegmentationandthe reg-istrationprocedure.Besidesanexplicitoptimizationofcorrespondence, the𝑐𝑜𝑚termalsoexploitslongitudinalinformationtoboostbothtasks, whichintroducessomedegreeofaugmentationandregularizationfor registrationandontheotherhandconstraintsandpriorknowledgefor segmentation.
Thehyperparameters𝛼 and𝛾 balancethelossmagnitudeof segmen-tation,registration,andtheirinterdependentcomposite.Thedegreeof regularizationonthedeformationisdescribedby𝛽.Theprocedureof simultaneousoptimizationissummarizedwithpseudocodein supple-mentarymaterial(Algorithm1).
3. ApplicationtodiffusionMRI
TheperformanceofSegis-Netisdemonstratedbyanalyzingwhite mattertractsinalargediffusionMRIdataset,andcomparedtothatof twomulti-stagepipelines,inwhichsegmentationandregistrationare in-dependentlyoptimized.Performanceisevaluatedinalongitudinal set-tingwheremultipletime-pointsfromthesameindividualareavailable.
3.1. Dataset
TheRotterdamStudyisaprospectiveandpopulation-basedstudy targetingcausesandconsequencesofage-relateddiseases(Ikrametal., 2020). Forthe present analysis, we included3249 individuals who underwentdiffusion MRIscanning twice or moreoften, resulting in
𝑁=8045scans.Themeanageatfirstscanwas61.2±9.4years(range: 45.7−91.1years).Thenumberoffemaleparticipantswas1780(54.8%). A flowchartforthe inclusion, exclusion,and splitof thedatasets is showninFig.2.Wesplitthedataintotwosubsets.Thelargersubsetwas repeatedlyacquiredinatimeintervalof1–5years(𝑁=7770scansfrom 3166individuals).Intheselongtime-intervalscans,itisexpectedthat brainmicrostructurechangesduetoagingexist.Bymatchinganytwo time-pointsfromthesameindividualregardlessof thevisitingorder, theselongtime-intervalscanscanbegroupedinto6043pairs.Weused 5175pairsofscansastrainingdata,200pairsasvalidationdatatotune thehyperparameters,monitorthedecayoflearningrateandselectthe optimalepoch,andusedanindependentcohortof668pairsfortesting. Theremainingscansfromthesmallersubsetwerefrom97individuals whowerescannedtwicewithinamonth.Nochangesinbrain macro-andmicrostructurewereexpectedwithinsuchashorttime-interval.We
Fig.2.Aflowchartfortheinclusion,exclusion,andsplitofthedatasets.
usedthesescansforevaluationofreproducibilityofthealgorithm.The datasplitwasbased ontheparticipants,namely,wemadesurethat scansfromthesameparticipantendedupineithertraining,validation, ortestdataset.
3.2. MRIacquisition
Scanswereacquiredona1.5TMRIscanner(GESignaExcite).The acquisitionparametersforstructuralanddiffusionMRIcanbefoundin
Ikrametal.(2011).Specifically,diffusionMRIwasscannedwiththe fol-lowingparameters:TR/TE=8575𝑚𝑠∕82.6𝑚𝑠,imagingmatrixof64× 96, FOV=21× 21𝑐𝑚2,35contiguoussliceswithslicethickness3.5mm,25
diffusionweightedvolumeswithab-valueof1000𝑠∕𝑚𝑚2 and3
non-weightedvolumes(b-value=0𝑠∕𝑚𝑚2).Thevoxelsizewasresampled
from3.3× 2.2× 3.5𝑚𝑚3to1𝑚𝑚3asrequiredforprobabilistic
tractogra-phy(Behrensetal.,2007).
3.3. Imagepreprocessing
Diffusion data were preprocessed using a standardized pipeline (Koppelmansetal.,2014).Inshort,motionandeddycurrentswere cor-rectedbyaffineco-registrationofalldiffusionweightedvolumestothe averagedb0volumes,includingcorrectionofgradientvectordirections usingElastixsoftware(Kleinetal.,2010).Diffusiontensorswere esti-matedwithaLevenberg–Marquardtnon-linearleast-squares optimiza-tionalgorithm(Leemansetal.,2009).WesubsequentlycomputedDTI measures: fractionalanisotropy(FA)andmeandiffusivity(MD).Due tonoise,tensorestimationfailedinasmallproportionof voxels, re-sultinginsignificantoutliers.Outliervoxelswithatensornorm (Frobe-niusnorm)largerthan0.1𝑚𝑚2∕𝑠weresettozero(Zhangetal.,2007).
BraintissuemasksincludingWMandgraymattersegmentationswere obtainedbasedonstructuralimaging(Vroomanetal.,2007) and ap-plied tothediffusiontensorimages.Inthisstudy,weuseda ROIof 112× 208× 112voxelstoanalyzesixWMtracts,includingleftandright cingulategyruspartofcingulum(CGC),leftandright parahippocam-pal partof cingulum(CGH),forcepsmajor(FMA)andforcepsminor (FMI).Diffusiontensorimageswereimage-wisenormalizedbysetting theunionofthesixcomponentstozero-meanandstandarddeviation ofone.Theaffinematrix(𝝓𝐴)ofeachimagepairwasestimatedby
op-timizingthemutualinformationofFAimagesusingElastixsoftware.
3.4. Referencesegmentations
Thesegmentationlabelsformodeltrainingandevaluationwere gen-eratedusingaprobabilistictractographyandatlas-basedsegmentation
methodbyDeGrootetal.(2015).Theresultingtract-densityimagesfor eachtractwerenormalizedbydivisionwiththetotalnumberoftracts inthetractographyrun.Finally,tract-specificthresholdsforthe normal-izeddensityimageswereestablishedbymaximizingthereproducibility ofFAmeasuresonasubsetof30participants(DeGrootetal.,2013b). Wedidnotexcludethissubsetfromthereproducibilitytestdata(Fig.2), asitremainsunseentotheproposedmethodandotherbaseline meth-ods.
3.5. Baselinemulti-stagepipelines
We compared the performance of the proposed Segis-Net with twomulti-stagepipelinesthatconsistofeithernon-learning-based or learning-basedalgorithmstoinvestigate theadded valueof simulta-neousoptimization.Toassesswhethertheperformancedifference be-tween approacheswas statistically significant, pairedt-tests with P-valuethreshold <0.05and Bonferronicorrection forcontrolling the family-wiseerrorofmultipletestingswereperformed.
First,anon-learning-basedClassicalpipelinewasbuiltusingan ex-istingtractography-basedsegmentationalgorithm(Section3.4)anda deformableregistrationalgorithmElastix(Kleinetal., 2010).Elastix wasadoptedasacompetingclassicalregistrationmethodsinceithas beenwidelyusedonourdatasetandtherebyanoptimalparameter set-tingcanbeappliedforperformancecomparison.Elastixisdesignedto runinacascadeofresolutions,andoffersthechoicebetween multi-pleobjectivefunctionsandmultipleoptimizersincludinganefficient adaptivestochasticgradientdescentoptimizer(Kleinetal.,2009).For Elastix(version4.8),weusedarigid,affine,andB-spline transforma-tionmodelconsecutivelybymaximizingmutualinformationbetween images.TheB-splinetransformationofsplineorder3wasimplemented usingamulti-resolutionframeworkwithisotropiccontrol-pointspacing of24,12,and6𝑚𝑚inthree-levelresolutions.Themaximumnumberof iterationswas1024.
Second,webuiltalearning-basedCNNpipelineusingcomponents fromtheproposedSegis-Nettoevaluatethesolecontributionof simul-taneousoptimization.Inthispipeline,wesplittheintegrated segmen-tation𝚯andregistrationstream𝚽intotwoseparateneuralnetworks forindependentoptimization.Subsequently,thesegmentedimagesand estimatedtransformationswerecombined.Thesegmentationnetwork hadthesamearchitectureasthatfor𝚯,exceptbeingindependently optimizedusingthesegmentationaccuracy𝑠𝑒𝑔term.Asthisisa typ-icalsetting forCNN-basedsegmentationapproaches(Lietal., 2018; Ronnebergeretal.,2015),wedenoteitasSeg-Net.Similarly,the regis-trationnetworkdenotedasReg-Nethadthesamesettingasthatfor𝚿, exceptbeingindependentlyoptimizedusingregistrationsimilarity𝑟𝑒𝑔
andregularization𝑑𝑒𝑓 terms(Balakrishnanetal.,2019).Weensured
thatthetrainingdatasetfortheSeg-NetandReg-Netwasthesameas thatusedfortheSegis-Netframework.
3.6. Relatedmethodsinvolvingsegmentationandregistration
Whenitcomestothecombinationofsegmentationandregistration, therearevariousintegrationstrategies(Section1).Toinvestigatethe benefitoftheproposedsimultaneousoptimizationstrategy, we addi-tionallycomparedSegis-Netwithtwopreviouslypublishedmethods:
• U-ReSNetforsimultaneoussegmentationandregistrationthatuseda sharedfeatureencoderandseparatedecoders(Estienneetal.,2019).
• VoxelMorph for image registration alone that used correspon-dence in existing segmentation labels to boost registration (Balakrishnanetal.,2019).
3.7. Implementation
ForthisdiffusionMRIapplication,thesegmentation(𝚯)and reg-istration (𝚿) components of Segis-Net used different input images.
Specifically,segmentationwasbasedonthediffusiontensorimage,asit containsdirectionalinformationoffiberpopulationsandwasshownto beoptimalinthepresentsettingofclinical-qualityresolution(Lietal., 2020a).Forspatialalignment,weadoptedtheinputthecommonlyused scalar-valueFAmapderivedfromdiffusiontensorimaging.
To mitigate class imbalance and to improve computational effi-ciency, we combined the reference segmentation for the six tracts (Section3.3)intoathree-channelmapforusingitasthesegmentation groundtruth 𝑆.Thiscombination waspossiblesinceonlyfew cross-ingfibersareexpectedbetweencodirectionalWMtracts(e.g.,FMIand FMA).Toevaluateperformanceonindividualtractsaftertraining,we extractedthetwolargestcomponentsfromeachofthethreechannels oftheprobabilisticpredictionandsubsequentlyidentifiedtheleftand right(forCGCandCGH)ortheanteriorandposterior(forFMIandFMA) tractbasedoncoordinates.
Theexperimentsofmodeltrainingandevaluationwereperformed onanNVIDIA1080TiGPUandanAMD1920XCPU.CNN-based meth-odswereimplementedusingKeras-2.2.0withaTensorflow-1.4.0 back-endandtheAdamoptimizer(KingmaandBa,2014).ForReg-Net, Seg-NetandSegis-Net,weightsofconvolutionkernelswereinitializedwith the Glorot uniform distribution (Glorot andBengio, 2010). In each training epoch,inputimageswerefedinrandombatches (size=1). Lossfunctionhyperparameterswereoptimizedbasedonsegmentation andregistrationperformanceonthevalidationdataset(searchrange: [10−3,10−2,10−1,100,101,102,103]);we setto 𝛼 =10, 𝛽 =0.01×𝛼 for Reg-Net;forSegis-Netwelinearlyincreased𝛼 from10to100by4per epoch (with𝛽 increasedaccordingly),andsettheadditional parame-ter𝛾 =1.Theinitiallearningrateswereexperimentallyoptimizedon thevalidationdatasetandsetto1𝑒−4,1𝑒−3and1𝑒−3forReg-Net,Seg-Net
andSegis-Net,whichweredecayedwithafactorof0.8ifthevalidation lossstoppeddecreasingfor10epochs(decaycondition,Algorithm1). Westoppedthetrainingprocedureatthepointthatthevalidationloss showedconsecutiveincreases,i.e.,earlystopping(Bishop,2006).The parametersofthemodelwiththesmallesterrorwithrespecttothe val-idationdatasetwereused.
For VoxelMorph, the implementation as detailed by
Balakrishnan et al. (2019) was used directly. For U-ResNet, in contrast to the other tensor-based segmentation methods, we used theFAmapasinputforbothsegmentationandregistrationsincethe sharedfeature-encoderrequiredthesameinputforbothtasks.Affine registration was applied as a pre-processing step. Hyperparameters were tuned on the validation dataset; and we obtained improved performancebyusinganinitiallearningrateof0.0005andbyclipping thewarpedsegmentationpredictionsintotherangeof[10−7,1−10−7]. 4. Experimentsandresults
WeappliedthemethodstoanalyzesixWMtracts.Theperformance oftheproposedSegis-Netwascomparedwiththetwobaseline multi-stagepipelinesonsegmentationaccuracy,registrationaccuracy, spatio-temporalconsistencyofsegmentation,reproducibilityofsegmentation andmeasurements,andsample-sizereduction;andcomparedwiththe tworelatedmethodsintermsofthesegmentationandregistration ac-curacy.
4.1. Segmentationaccuracy
Segmentationaccuracywasquantifiedwithrespecttothereference segmentation(Section3.4)usingtheDicecoefficientmetric.
Theproposedmethodyieldedsimilarsegmentationaccuracyasthe baseline multistage CNNpipeline (Seg-Net) forall six tracts(Fig.3). Bothmethodsachievedrelativelyhighaccuracyinsegmenting cingu-lum,i.e.,theaccuracyofleftandrightCGCandCGHtractswasaround 0.76±0.07.TheaccuracywaslowestforFMI(CNN:0.68±0.09; Segis-Net:0.67±0.09),whichisathinandarch-shapedtractthatisknown tobe moredifficulttosegment. Correctingfor6tests resultedin an
Fig.3.SegmentationaccuraciesoftheCNNpipelineandSegis-Netfordifferent tracts.Errorbarsindicatestandarddeviations.
Table1
SegmentationDicecoefficientofU-ReSNetandSegis-Net.Theboldvalue indi-catesabetterperformanceineachrow.
U-ReSNet Segis-Net CGC_L 0 . 69 ± 0 . 06 0.76 ± 0.06 CGC_R 0 . 69 ± 0 . 07 0.76 ± 0.06 CGH_L 0 . 67 ± 0 . 08 0.76 ± 0.07 CGH_R 0 . 67 ± 0 . 09 0.76 ± 0.09 FMA 0 . 69 ± 0 . 06 0.76 ± 0.05 FMI 0 . 60 ± 0 . 08 0.67 ± 0.09
adjustedP-valuethresholdof8.3× 10−3.Therewasnosignificant
differ-encesinsegmentationaccuracybetweentwomethods.
Theproposedmethodshowedhigher segmentationaccuracythan U-ReSNetforallsixtractswithamarginofaround10%andsmaller standarddeviation(Table1).
4.2. Registrationaccuracy
Registrationaccuracyoftheapproacheswasevaluatedwiththe spa-tialcorrelation(SC)similarityonthetestdataset.Accordingtothe pro-cedure inDeGrootet al.(2013b), theestimatedtransformationwas appliedtothecontinuousdensity mapsofindividualtracts obtained fromprobabilistictractography,subsequently,theSCsimilaritybetween warpeddensitymapswascomputedasfollow:
𝑆𝐶𝑘= ∑ 𝑥∈Ω𝑡𝐽𝑡𝑘(𝐽𝑠𝑘◦ ̂𝝓) ( ∑ 𝑥∈Ω𝑡 √ (𝐽𝑡𝑘)2 )( ∑ 𝑥∈Ω𝑡 √ (𝐽𝑠𝑘◦ ̂𝝓)2 ), (12) where𝐽𝑘
𝑡 and𝐽𝑠𝑘indicateintensityofthetargetandsourcedensity
im-ageofthetract𝑘.Despitealotofintensityvariationinthetract den-sitymapsacrossscansduetotheprobabilisticnatureoftractography, higherintensityin generalindicatesmoresupportforthetract while lowerintensityconverselyindicatesincreaseduncertainty.Therefore, weassumethatSCreflectsthespatialcorrespondenceoftracts.
Fig.4presents theregistrationaccuracy(SC)ofSegis-Netandthe baselinemultistagepipelines.TheSCinallsixtractswasoverall high-estfortheSegis-Net,followedbytheClassicalpipeline.Correctingfor 18testsresultedinBonferroniadjustedP-valuethresholdof2.8× 10−3.
Segis-Netresultsyieldedasignificantlybetterspatialcorrespondence than theClassical pipelinein the left CGH (Segis-Netvs Classical = 0.77±0.09 vs 0.75±0.11), FMA (0.74±0.09 vs 0.72±0.10),and FMI (0.76±0.08vs0.74±0.08)tract.Statisticallysignificantdifferenceinthe registrationaccuracyofSegis-NetandCNNpipelinewereobservedin theleftCGC(Segis-NetvsCNN=0.73±0.08vs0.71±0.07),rightCGC (0.73±0.07vs0.69±0.06),leftCGH(0.77±0.09 vs0.75±0.08),right
Fig.4.RegistrationaccuraciesoftheClassical,CNN,andSegis-Netpipelineas quantifiedbyspatialcorrelation(SC)oftheregisteredtractdensitymaps.Error barsindicatestandarddeviations.Thebrackethatindicatesasignificant differ-encebetweentwomethods(t-test,𝑝<2.8× 10−3).
Table2
RegistrationperformanceofU-ReSNet,VoxelMorphandSegis-Net,asquantified bythespatialcorrelation(SC)similarity,theDicecoefficient(DC),andthemean squarederror(MSE).Theboldvalueindicatesthebestperformanceineachrow.
U-ReSNet VoxelMorph Segis-Net
SC CGC_L 0.77 ± 0.11 0 . 72 ± 0 . 09 0 . 73 ± 0 . 08 CGC_R 0.77 ± 0.11 0 . 71 ± 0 . 09 0 . 73 ± 0 . 07 CGH_L 0.77 ± 0.11 0 . 75 ± 0 . 10 0.77 ± 0.10 CGH_R 0.77 ± 0.12 0 . 75 ± 0 . 11 0.76 ± 0.10 FMA 0.73 ± 0.11 0 . 73 ± 0 . 10 0.74 ± 0.09 FMI 0.75 ± 0.09 0 . 74 ± 0 . 08 0.76 ± 0.08 DC CGC_L 0 . 69 ± 0 . 07 0 . 65 ± 0 . 06 0.74 ± 0.06 CGC_R 0 . 70 ± 0 . 07 0 . 65 ± 0 . 06 0.74 ± 0.05 CGH_L 0 . 67 ± 0 . 08 0 . 64 ± 0 . 07 0.71 ± 0.08 CGH_R 0 . 67 ± 0 . 10 0 . 64 ± 0 . 08 0.71 ± 0.09 FMA 0 . 70 ± 0 . 06 0 . 68 ± 0 . 06 0.72 ± 0.06 FMI 0 . 57 ± 0 . 10 0 . 56 ± 0 . 07 0.60 ± 0.09 MSE ( ×10 −2 ) 0 . 47 ± 0 . 38 0 . 19 ± 0 . 88 0.13 ± 0.10 CGH(0.76±0.10vs0.75±0.10),andFMA(0.74±0.09vs0.71±0.08) tract.Ingeneral,theproposedSegis-Netapproachachievedabetter spa-tialcorrespondencethanthetwoindependentlyoptimizedregistration algorithmsusingclassicalandlearning-basedtechniques.
For comparing the proposed method with U-ReSNet and Voxel-Morph,weaddedtheperformancemetricthatwasusedintheiroriginal papers(Balakrishnanetal.,2019;Estienneetal.,2019),i.e.,theDice co-efficient(DC)ofregisteredreferencesegmentationoftracts,andadded thecommonlossmetric,i.e.,themeansquarederror(MSE)between reg-isteredFAmaps(Table2).Generally,theSCsimilarityoftheproposed methodandU-ReSNetwerebetterthanthatofVoxelMorph.Segis-Net ledtothehighestsimilarityinFMAandFMItract;U-ReSNetwasthe highestfortheleftandrightCGC,andtherightofCGHtract;forthe leftCGHtract,asimilarSCwasobservedforU-ReSNetandSegis-Net, althoughthevariationsweresmallerinSegis-Net; Segis-Netachieved thebestDCandMSE.Forallsixtracts,theDCofSegis-Netwerehigher thanthatofU-ReSNet,followedbyVoxelMorph;thestandarddeviation ofSegis-Netwasoverallsmallestforallthreemetrics,exceptthatofDC inthreetracts(leftandrightCGH,andFMI)whichweresmallestfor VoxelMorph.
4.3. Spatio-temporalconsistencyofsegmentation
Toevaluatethespatio-temporalconsistencyofsegmentation(STCS) forSegis-Netandthebaselinemultistagepipelines,wemeasuredthe correspondence between warped segmentation results across time-pointsusingtheDicecoefficient.Theconsistencyofeachtractwas
av-Fig.5. Spatio-temporalconsistencyof segmentation(STCS) withthe Classi-cal,CNN,andSegis-Netpipeline.Errorbarsindicatestandarddeviations.The bracket hat indicates a significantdifferencebetween two methods (t-test, 𝑝<2.8× 10−3).
eragedovertwodirectionsbyreversingthetargetandsourceimage, whichforthetract𝑘canbeformulatedas:
𝑆𝑇𝐶𝑆𝑘=12 (2| ̂𝑆𝑘 𝑡 ∩ ̂𝑆𝑠𝑘◦ ̂𝝓| | ̂𝑆𝑘 𝑡| +| ̂𝑆𝑠𝑘◦ ̂𝝓| + 2| ̂𝑆 𝑘 𝑠∩ ̂𝑆𝑡𝑘◦ ̂𝝓−1| | ̂𝑆𝑘 𝑠| +| ̂𝑆𝑡𝑘◦ ̂𝝓−1| ) . (13)
Each pipeline was evaluated as a whole, that is, (1)in Classical
pipelinethereferencesegmentationwaswarpedbyElastixalgorithm, (2)inCNNpipelinethepredictionofSeg-Netwaswarpedbythe pre-dictedtransformationoftheReg-Net,and(3)inSegis-Netframeworkthe segmentationpredictioninnativespaceandthesegmentationwarped fromanothertime-pointwereavailableafterabidirectionaltest.
Theproposed Segis-Netoverallshowedhigher segmentation con-sistencythantheCNNandtheClassicalpipeline(Fig.5).Correcting for18tests resultedinanadjustedP-valuethresholdof2.8× 10−3.In
comparison with theCNNpipeline, Segis-Net results yielded signifi-cantlyhigherspatio-temporalconsistencyinleftCGC(Segis-NetvsCNN
=0.83±0.04vs0.82±0.04),rightCGC(0.83±0.04vs0.82±0.06),left CGH(0.82±0.05vs0.81±0.05),FMA(0.87±0.02vs0.84±0.03),and FMI (0.81±0.05 vs 0.77±0.07) tract.In all six tracts, Segis-Net sig-nificantlyoutperformedtheClassicalpipeline,i.e.,inleftCGC (Segis-NetvsClassical=0.83±0.04vs0.68±0.06),rightCGC(0.83±0.04vs 0.68±0.06),leftCGH(0.82±0.05vs0.66±0.08),rightCGH(0.81±0.05 vs0.66±0.09),FMA(0.87±0.02vs0.69±0.06),andFMI(0.81±0.05vs 0.57±0.09)tract.
4.4. Reproducibilityofsegmentationandmeasurements
Reproducibilityoftract-specificsegmentations,volumes,and diffu-sionmetricsofthepipelines wasevaluated usingthereproducibility dataset.Wequantifiedvoxel-wiseagreementbetweensegmentationsof repeatedscansusingCohen’skappacoefficient(𝜅).Thesegmentations (̂𝑡,̂𝑠)wereobtained inthenativespace,andsubsequentlyaligned (̂𝑠◦ ̂𝝓).Kappa𝜅 ofthetract𝑘isdefinedas:
𝜅𝑘= 𝑝𝑜
(̂𝑆𝑡𝑘, ̂𝑆𝑠𝑘◦ ̂𝝓)−𝑝𝑒(̂𝑆𝑡𝑘,̂𝑆𝑠𝑘◦ ̂𝝓)
1−𝑝𝑒(̂𝑆𝑡𝑘, ̂𝑆𝑠𝑘◦ ̂𝝓) , (14)
inwhich𝑝𝑜(̂𝑆𝑡𝑘, ̂𝑆𝑠𝑘◦ ̂𝝓)istheobservedagreementbetween ̂𝑆𝑡𝑘and ̂𝑆𝑠𝑘◦ ̂𝝓 ,
𝑝𝑒is thehypotheticalprobabilityof theagreement.Given|Ω𝑡| being
thetotalnumberofvoxelsinthetargetimage,|𝑆| and|Ω𝑡| −|𝑆| being
thenumberoftractandnon-tractvoxels,theobservedagreement(i.e., accuracy)iscomputedas:
𝑝𝑜(̂𝑆𝑡𝑘, ̂𝑆𝑠𝑘◦ ̂𝝓)= | ̂𝑆𝑘 𝑡 ∩ (̂𝑆𝑠𝑘◦ ̂𝝓)| +|(1− ̂𝑆𝑡𝑘)∩ ( 1−(̂𝑆𝑠𝑘◦ ̂𝝓))| |Ω𝑡| , (15)
thehypotheticalprobabilityoftheagreementcanbeformulatedas:
𝑝𝑒(̂𝑆𝑡𝑘, ̂𝑆𝑠𝑘◦ ̂𝝓)= |Ω1 𝑡|2 ( | ̂𝑆𝑘 𝑡| × | ̂𝑆𝑠𝑘◦ ̂𝝓| +(|Ω𝑡| −| ̂𝑆𝑡𝑘|)× (|Ω𝑡| −| ̂𝑆𝑠𝑘◦ ̂𝝓|) ) . (16) Typically,a𝜅 >0.60indicates“substantial” agreement,anda𝜅 >0.80 indicates“almostperfect” agreement(LandisandKoch,1977).
Similarly,toevaluatethereproducibilityoftract-specific measure-ments,wecomputedtheFA,MDandvolumeinimagenativespace,and subsequentlyassessedrelativedifferences in pairedscan-rescan mea-sures(𝑚𝑡,𝑚𝑠)asanindicatorofmeasurementerror(𝜖),i.e.,
𝜖 = 2(|𝑚𝑚𝑠−𝑚𝑡|
𝑠+𝑚𝑡) × 100%. (17)
ForFAandMD,thetract-specificmeasureswerequantifiedasthe me-dianofnon-zerovalueswithinthesegmentedimages.Alower𝜖
indi-catesabetterreproducibility.
Fig.6presentsthereproducibilityoftract-specificsegmentationand measuresdeterminedwiththebaselinemulti-stagepipelinesand Segis-Net. The proposed Segis-Net achieved the best segmentation repro-ducibility,followedbytheCNNpipeline(Fig.6(a));inallsixtracts,𝜅 wasaround0.80orhigher,indicating“almostperfect” agreements be-tweensegmentationsofrepeatedscans.Correctingfor18testsforeach metricresulted inanadjustedP-valuethresholdof 2.8× 10−3, result-inginoverallstatisticallysignificantimprovementbySegis-Netoverthe
Classicalpipeline.Fortwotracts,voxel-wiseagreementofSegis-Netwas significantlyhigherthanthatoftheCNNpipeline,i.e.,FMA(Segis-Net vsCNN=0.87±0.03vs0.85±0.03)andFMI(0.82±0.06vs0.79±0.08). Additionally,intheevaluationofthereproducibilityintract-specific volumemeasures,Segis-Netshowedthesmallesterrorinallsixtracts (Fig.6(b)).TheerrorofSegis-Netwassignificantlysmallerthanthe
Classical pipeline in left CGC (Segis-Net vs Classical =4.8±4.1% vs 11±8.8%), rightCGC (4.5±3.9% vs 11±8.9%),left CGH(7.3±5.6% vs 11±9.8%),andFMA (3.4±2.6% vs6.6±5.9%) tract. This outper-formedtheCNNpipelinesignificantlyintheFMAtract (Segis-Netvs
CNN=3.4±2.6%vs4.9±3.6%).Reproducibilityof FAandMD mea-surementswassimilarforthethreemethods(Fig.6(c,d)).FortheCGH andleftCGCtracts,thereproducibilityofFAusingtheCNNpipeline was significantlyhigherthanthat oftheClassicalpipeline.Segis-Net outperformedtheFAreproducibilityoftheClassicalpipelineonlyinthe leftCGCtract.ForMD,nosignificantimprovementovertheClassical
pipelinewasobserved.Atable(Table3)withtheresultsofFig.3–6is providedinthesupplementaryfiles.
4.5. Sample-sizereduction
An implication of the reduced measurement error (𝜖) is that fewer participants or time-points would be required to achieve the same statistical power, i.e., a smaller sample size. We followed
Diggleetal.(2002)andReuteretal.(2012)toestimatethepercentage ofthesamplesizes(𝑃)thatwouldberequiredforeachofthepipelines:
𝑃𝑖𝑗= 𝜎 2 𝑖× (1−𝜌𝑖) 𝜎2 𝑗 × (1−𝜌𝑗) × 100%, (18)
where𝜎𝑖and𝜎𝑗 arestandarddeviationsin themeasurements
deter-minedwiththepipeline𝑖and𝑗,and𝜌𝑖and𝜌𝑗arethecorrelation
coef-ficientsbetweentherepeatedmeasurementsdeterminedwiththetwo pipelines.
Fig.7presentsthepercentageofsample-sizereductionthatcouldbe achievedbytheCNNandtheproposedSegis-Netcomparedtothe Clas-sicalpipeline.Inlinewiththereproducibilityresults,thedataanalyzed withSegis-Netwouldoverallrequiretheleastsample-sizetoachieve thesamestatisticalpower.Thepercentageofreductionwasespecially remarkableinvolumemeasures,inwhichonaverageonly33.0%ofdata wouldberequired.Theaveragepercentageofreductionwas60.5%for
Fig.6. Reproducibilityoftract-specificmeasureswiththeClassical,CNN,andSegis-Netpipeline.Errorbarsindicatestandarddeviations.Thebrackethatindicates asignificantdifferencebetweentwomethods(t-test,𝑝<2.8× 10−3).Infigure(a),ahigherCohen’skappacoefficient(𝜅)indicatesabetterreproducibility.Infigure (b-d),alowererror(𝜖%)indicatesabetterreproducibility.Volume:tract-specificvolume(ml),FA:fractionalanisotropy,MD:meandiffusivity(10−3𝑚𝑚2∕𝑠).
Fig.7. Thepercentageofsample-sizethatwouldberequiredintractmeasuresofvolume,FA,andMDwiththeCNNpipelineandSegis-Net.Thesample-sizerequired fortheClassicalpipelineisusedasthereference(100%).
FAand57.0%forMD.SeveralpercentagesoftheCNNpipelinewere smallerthanthoseoftheSegis-Net,e.g.,inFAmeasuresofCGHand FMAtract(Fig.7(b)),butitsperformanceshowedtobelessstableacross tractsthantheSegis-Net,whichinallsettingsconsistentlydecreasedin therequiredsamplesovertheClassicalpipeline.Thepercentageof re-ductionwasgenerallysimilarforleft/righthomologoustractsexceptfor theMDmeasureintheleftofCGH(Fig.7(c)).Thislargereductioncould berelatedtotheMDreproducibilityoftheClassicalpipeline,inwhich theleftCGHtracthadamuchhighervariationinerrorscomparingwith thatoftheothertracts(Fig.6(d)).
5. Discussion
Wedevelopedasingle-stepdeeplearningframework,coined Segis-Net, forsimultaneousoptimization of segmentationandregistration. ThemethodwasappliedtoanalyzechangesinWMtractsfromalarge setoflongitudinaldiffusionMRIimages.Toevaluatetheperformance ofthemethod,wecompareditwithtwostate-of-the-artmethods,and twomultistage pipelinesconsistingofindependentsegmentationand registrationcomponents,i.e.,theClassicalandCNNpipeline.Segis-Net advancedthestate-of-the-artbyahighersegmentationandregistration
accuracy,andledtoimprovedperformancesinregistrationaccuracy, spatio-temporalconsistencyofsegmentation,andreproducibilityof seg-mentationandtract-specificmeasurescomparingwiththemulti-stage pipelines.Weevaluatedthepracticalvalueoftheimprovedperformance in termsof sample-size reductionthat could be achievedwhen em-ployingthemethod.Thetract-specificmesuresanalyzedwithSegis-Net wouldonlyrequire33.0%−60.5%sample-sizeofthedataforachieving thesameeffectsizeastheClassicalpipeline.
Todatemostdevelopmentsinlongitudinalanalysisframeworkshave focusedonunbiasedwaysofregisteringimagetimeseries(Keihaninejad etal.,2013;Metzetal.,2011),inwhichamultistageapproach comb-ingindependentsegmentationandregistrationcomponentsisoftenused (DeGrootetal.,2013a;Yendikietal.,2016).Inthispaper,weaimed toinvestigateadifferentwaytoimprovetheperformanceofthe lon-gitudinalframeworkbyusinga single-stepCNNthat optimizesboth taskssimultaneously.Thesolevalueofsimultaneousoptimizationwas demonstratedbythecomparisonwiththeCNNpipeline.Therewasno benefitobserved forsegmentationalone,butfor registration, spatio-temporalconsistencyofsegmentation,andreproducibility, simultane-ousoptimizationledtosignificantlyimprovedperformance.
Intheevaluationofsegmentationperformance,similaraccuracies fortheCNNandSegis-Netframeworkwasobservedforthesixtracts (Fig.3).Relativesegmentationaccuracybetweenindividualtractswere inlinewiththosereportedin literature(Lietal., 2020a;Wasserthal etal.,2018).Forinstance,asmallandcurvedobjectliketheFMItract tendedtohavealowerDicecoefficientthanthelargerFMAtracts.For allsixtracts,theproposedSegis-Netshowedabettersegmentation per-formancethanU-ReSNet,anexistingsimultaneousmethod(Table1). WeexpecttheaddedvalueofSegis-Nettoberelatedtotwofactors:(1) themethodallowstheuseofdiffusiontensorimagesfortract segmen-tation,asweuseparallelnetworkmodulesandonlyalignthepredicted segmentation;thiscircumventstheneedtointerpolatetensorimages.In otherwords,task-specificinputscanbeused;and(2)thesub-branches inthesegmentationstream(Fig.8)aredesignedforthepredictionof whitemattertractswhichcanoverlapwitheachother,unlikethe ex-clusivetissuelabelsfocusedbyotherworks.
Inthetaskofregistrationalone,Segis-Netoverallyieldedthebest accuracyamongthemethods.Itsignificantlyoutperformedthe Classi-calpipelineforthreetractsandtheCNNpipelineforfivetracts(Fig.4). Thisisanimportantobservationas(1)itshowedthatsimultaneous op-timizationwasbeneficial tooneof theindividualtasks, and(2)itis non-trivialtoimproveregistrationaccuracyoveraclassicalalgorithm, inwhichthetransformationispair-wiseoptimizedonthetestimages. Duringthecomparisonwiththestate-of-the-artmethods,weobserved twointerestingresults(Table2).First,VoxelMorphwastheonlymethod thatdirectlyoptimizedontheDCmetric,butitledtoalowestDCscore. Thiscanbeduetothefactthatthesegmentationlabelsusedindiffusion imagingstudiesareoftenindependentlyobtainedforeachimage,which ismuchlesscorrelatedtotheregistrationperformancethanisthecase foratlas-basedsegmentation(Balakrishnanetal.,2019).Asaresult,the alignmentof“imperfect” segmentationlabelscanbeanobstructiveloss terminstead.Second,althoughtheMSEofU-ReSNetwasalmostfour timesthatoftheSegis-Net,itachievedagoodSCsimilarity,especially inthethinstructures(CGCandCGH).Thiscanbeattributedtothe for-mulationoftheirregistrationlossasthesumoflocalcross-correlation andMSE.
Inallsix tracts,we observedsubstantiallyhigher spatio-temporal consistencyofsegmentationandreproducibilityofsegmentationwith Segis-Netthanwiththetwomultistagepipelines(Figs.5,6).The spatio-temporalconsistency of segmentation as quantified by the Dice co-efficient ranged0.81−0.87for Segis-Net, significantlyoutperforming theClassicalpipelineforallthesixtracts(range:0.57−0.69)andthe
CNNpipelineforfivetracts(range:0.77−0.84).Thesegmentation re-producibility as quantified by Cohen’s kappa ranged 0.79−0.87 for Segis-Net,significantlyhigherthantheClassicalpipelineforallthesix tracts(range:0.64−0.72)andtheCNNpipelinefortwotracts(range:
0.77−0.85).These resultsindicate thatSegis-Netcan serve asa reli-ablealternativetotheClassicalpipelineinspatiallycapturing macro-structuralbrainchangesovertime.
Inaddition,moresignificantimprovementswereobservedforthe re-producibilityoftract-specificvolumeassessment,butnotfortheFAand MDmeasures.Forvolumereproducibility,Segis-Netyieldedtheleast error inthemeasurementsofscanandre-scan,followedbytheCNN
pipeline(Fig.6).FortheFAandMDmeasures,weobservedrelatively similarreproducibilityforthethreemethods,inwhichsignificant differ-encewasonlyobservedinFAreproducibilityofCGHandleftCGCtract. Thissuggeststhatdiffusionmeasuresarequiterobusttovariationsin thegeometryofthesegmentedtract.It’sworthnotingthattheFA repro-ducibilityoftheClassicalpipelinecouldbehigherthanthebenchmark oftractography-basedsegmentationmethods,sinceitisoptimizedon theFAreproducibilityonasubsetofthedata.
Theseimprovedperformanceshavepracticalvaluesinapower anal-ysis,whereboththeCNNpipelineandSegis-Netshowedtobeableto reducetherequiredsample-sizetoachievethesamestatisticalpoweras theClassicalpipeline.ThedataprocessedwithSegis-Netwouldrequire onaverage33.0%ofthesample-sizeforvolumemeasures,60.5%forFA, and57.0%forMDmeasures,requiringconsistentlyadecreased sample-sizeforallthesettings.TheaveragedpercentagesfortheCNNpipeline were62.7%,60.5%and68.7%.ForFMItract,itwould,however,require 183%and124%ofthesample-sizeforthevolumeandFAmeasures.The observeddispersionofsample-sizereductionwiththeCNNpipelinemay suggestthatsimultaneousoptimizationwasbeneficialtotherobustness ofthemethodacrosstheconcurrentlysegmentedtracts.
Whereas themethod isgeneric, wespecifically implemented and optimizeditforlongitudinalstudyindiffusionMRIdata.Indiffusion MRIapplication,weadoptedthecommonlyusedscalar-valueFAmap astheinputforregistration.Deformableregistrationofdiffusiontensor imagesisknowntobechallengingduetothedirectionalcomponents containedinvoxels.Despitedevelopmentsinclassicalmethodsfor ten-sorreorientationduringtheoptimization(Caoetal.,2006;Zhangetal., 2007),forlearning-basedregistrationitstilllargelyremainsunexplored. Withthepromisingresultsofdiffusiontensorinterpolationasshownby
Grigorescuetal.(2020),Segis-Netbasedonsolelytensorimageswould beaninterestingdirectiontoexplore.
TheSegis-Netframeworkpresentedinthecurrentstudyislimitedto twotime-points.Thisisbecauselearning-basedregistrationalgorithms currently only supportpairwisetransformations (Balakrishnan etal., 2019).Onelimitationofourmethodisthereforethatitdoesnotallow foranalysisof arbitrarynumberof time-points.Inthepresentstudy, wegroupedtheavailabletripletime-pointsfromthesameparticipant intoorderlessimage-pairsforbidirectionalanalysis.Afuturepossible improvementofthemethodcouldbeextendingtheregistration compo-nentofSegis-Nettoenablelearning-basedgroup-wiseanalysisofaset oftime-points(Lietal.,2020b).
Beyondthecurrentapplication,weexpectthatthisworkcouldbe extended toother imagingsequencesandforexamplefor segmenta-tionoflesionimages.Forfuturework,weplantoadapttheproposed methodtoanalyzebraindiseaseswithlargeandprogressivechanges. Forinstance,registrationofbrainswithlesionsduetocorticalinfarct maybenefitfromasimultaneoussegmentationofinfarctregions.
6. Conclusion
Weproposedasingle-stepdeeplearningframeworkfor longitudi-naldiffusionMRIanalysis,inwhichsegmentationanddeformable reg-istrationwereintegratedforsimultaneousoptimization.The compari-sonwithbaselinemultistageapproachesandstate-of-the-artmethods showedthattheproposedSegis-Netcan beappliedasareliabletool tosupportspatio-temporalanalysisofWMtractsfromlongitudinal dif-fusionMRIimaging.Besidestheimprovedperformances,atwo-in-one frameworkforconcurrentsegmentationandregistrationalsoenablesa light-weightwayoffastquantificationofbrainchangesovertime.This
mayleadtoamoreprominentrolefortract-specificbiomarkersin ap-plicationswheretractsegmentationandregistrationaresubjecttotime constraints.With theincreasingavailabilityof longitudinal diffusion data,weexpectfuturestudiesinvestigatingprogressive neurodegener-ationcangreatlybenefitfromtheimprovedreliabilityandefficiencyof Segis-Net.
Dataavailability
Thedatasetsanalyzedduringthecurrentstudyarenotpublicly avail-able.Duetothesensitivenatureofthedatausedinthisstudy, partici-pantswereassuredrawdatawouldremainconfidentialandwouldnot beshared.
Codeavailability
ThecodeforSegis-Net,theCNNpipeline,aswellasthe implemen-tationofElastixisavailableathttps://gitlab.com/blibli/segis-net.
Ethicsstatement
TheRotterdamStudyhasbeenapprovedbytheMedicalEthics Com-mitteeoftheErasmusMC(registrationnumberMEC02.1015)andby theDutchMinistryofHealth,WelfareandSport(PopulationScreening ActWBO, licensenumber1071272-159521-PG).All participants pro-videdwritteninformedconsenttoparticipateinthestudyandtohave theirinformationobtainedfromtreatingphysicians(Ikrametal.,2020).
Creditauthorshipcontributionstatement
BoLi:Conceptualization,Methodology,Software,Formalanalysis, Validation,Writing-originaldraft,Writing-review&editing, Visual-ization.WiroJ.Niessen:Conceptualization,Validation,Resources, Su-pervision,Writing-review&editing,Fundingacquisition.StefanKlein:
Methodology,Software,Writing-review&editing.MariusdeGroot:
Software,Writing-review&editing.M.ArfanIkram:Resources,Data curation,Writing-review&editing.MeikeW.Vernooij:Resources, In-vestigation,Datacuration,Writing-review&editing.EstherE.Bron:
Formalanalysis,Investigation,Validation,Writing-review&editing, Supervision,Projectadministration.
Acknowlgedgments
ThisworkwassponsoredthroughgrantsoftheMedicalDelta Di-agnostics3.0:DementiaandStroke,theEUHorizon2020project Eu-roPOND (666992), the Netherlands CardioVascular Research Initia-tive(Heart-BrainConnection:CVON2012-06,CVON2018-28),andthe DutchHeartFoundation(PPPAllowance,2018B011).
SupplementaryMaterials
Supplementarymaterialassociatedwiththisarticlecanbefound,in theonlineversion,atdoi:10.1016/j.neuroimage.2021.118004.
References
Ashburner, J. , Friston, K.J. , 2005. Unified segmentation. Neuroimage 26 (3), 839–851 . Balakrishnan, G. , Zhao, A. , Sabuncu, M.R. , Guttag, J. , Dalca, A.V. , 2019. VoxelMorph:
a learning framework for deformable medical image registration. IEEE Trans. Med. Imag. .
Barrick, T.R. , Charlton, R.A. , Clark, C.A. , Markus, H.S. , 2010. White matter structural decline in normal ageing: a prospective longitudinal study using tract-based spatial statistics. Neuroimage 51 (2), 565–577 .
Bastiaansen, W.A. , Rousian, M. , Steegers-Theunissen, R.P. , Niessen, W.J. , Koning, A. , Klein, S. , 2020. Towards segmentation and spatial alignment of the human embryonic brain using deep learning for atlas-based registration. In: Proceedings of the Interna- tional Workshop on Biomedical Image Registration. Springer, pp. 34–43 .
Behrens, T.E. , Berg, H.J. , Jbabdi, S. , Rushworth, M.F. , Woolrich, M.W. , 2007. Probabilistic diffusion tractography with multiple fibre orientations: what can we gain? Neuroim- age 34 (1), 144–155 .
Bishop, C.M. , 2006. Pattern Recognition and Machine Learning. Springer .
Bortsova, G. , Dubost, F. , Hogeweg, L. , Katramados, I. , de Bruijne, M. , 2019. Semi-super- vised medical image segmentation via learning consistency under transformations. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 810–818 .
Cao, Y. , Miller, M.I. , Mori, S. , Winslow, R.L. , Younes, L. , 2006. Diffeomorphic matching of diffusion tensor images. In: Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06). IEEE, p. 67 .
Cheng, J. , Tsai, Y.-H. , Wang, S. , Yang, M.-H. , 2017. Segflow: Joint learning for video object segmentation and optical flow. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 686–695 .
Conturo, T.E. , Lori, N.F. , Cull, T.S. , Akbudak, E. , Snyder, A.Z. , Shimony, J.S. , McK- instry, R.C. , Burton, H. , Raichle, M.E. , 1999. Tracking neuronal fiber pathways in the living human brain. Proc. Natl. Acad. Sci. 96 (18), 10422–10427 .
Crick, F. , Jones, E. , 1993. Backwardness of human neuroanatomy.. Nature 361 (6408), 109–110 .
Crum, W.R. , Camara, O. , Hill, D.L. , 2006. Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Trans. Med. Imaging 25 (11), 1451–1461 . Dai, X. , Khorram, S. , 1999. A feature-based image registration algorithm using improved chain-code representation combined with invariant moments. IEEE Trans. Geosci. Re- mote Sens. 37 (5), 2351–2362 .
Dice, L.R. , 1945. Measures of the amount of ecologic association between species. Ecology 26 (3), 297–302 .
Diggle, P. , Diggle, P.J. , Heagerty, P. , Liang, K.-Y. , Heagerty, P.J. , Zeger, S. , et al. , 2002. Analysis of Longitudinal Data. Oxford University Press .
Dimond, D. , Rohr, C.S. , Smith, R.E. , Dhollander, T. , Cho, I. , Lebel, C. , Dewey, D. , Con- nelly, A. , Bray, S. , 2020. Early childhood development of white matter fiber density and morphology. Neuroimage 210, 116552 .
Estienne, T. , Lerousseau, M. , Vakalopoulou, M. , Alvarez Andres, E. , Battistella, E. , Carré, A. , Chandra, S. , Christodoulidis, S. , Sahasrabudhe, M. , Sun, R. , et al. , 2020. Deep learning-based concurrent brain registration and tumor segmentation. Front Comput. Neurosci. 14, 17 .
Estienne, T. , Vakalopoulou, M. , Christodoulidis, S. , Battistela, E. , Lerousseau, M. , Carre, A. , Klausner, G. , Sun, R. , Robert, C. , Mougiakakou, S. , et al. , 2019. U-ReSNet: ultimate coupling of registration and segmentation with deep nets. In: Proceedings of the Inter- national Conference on Medical Image Computing and Computer-Assisted Interven- tion. Springer, pp. 310–319 .
Farbota, K.D. , Bendlin, B.B. , Alexander, A.L. , Rowley, H.A. , Dempsey, R.J. , Johnson, S.C. , 2012. Longitudinal diffusion tensor imaging and neuropsychological correlates in traumatic brain injury patients. Front Hum. Neurosci. 6, 160 .
Fischl, B. , Salat, D.H. , Busa, E. , Albert, M. , Dieterich, M. , Haselgrove, C. , Van Der Kouwe, A. , Killiany, R. , Kennedy, D. , Klaveness, S. , et al. , 2002. Whole brain segmen- tation: automated labeling of neuroanatomical structures in the human brain. Neuron 33 (3), 341–355 .
Glorot, X. , Bengio, Y. , 2010. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Arti- ficial Intelligence and Statistics, pp. 249–256 .
Gooya, A. , Pohl, K.M. , Bilello, M. , Biros, G. , Davatzikos, C. , 2011. Joint segmentation and deformable registration of brain scans guided by a tumor growth model. In: Proceed- ings of the International Conference on Medical Image Computing and Computer-As- sisted Intervention. Springer, pp. 532–540 .
Grigorescu, I. , Uus, A. , Christiaens, D. , Cordero-Grande, L. , Hutter, J. , Edwards, A.D. , Ha- jnal, J.V. , Modat, M. , Deprez, M. , 2020. Diffusion tensor driven image registration: a deep learning approach. In: Proceedings of the International Workshop on Biomedical Image Registration. Springer, pp. 131–140 .
De Groot, M. , Cremers, L.G. , Ikram, M.A. , Hofman, A. , Krestin, G.P. , van der Lugt, A. , Niessen, W.J. , Vernooij, M.W. , 2016. White matter degeneration with aging: longitu- dinal diffusion MR imaging analysis. Radiology 279 (2), 532–541 .
De Groot, M. , Ikram, M.A. , Akoudad, S. , Krestin, G.P. , Hofman, A. , van der Lugt, A. , Niessen, W.J. , Vernooij, M.W. , 2015. Tract-specific white matter degeneration in ag- ing: the rotterdam study. Alzheimer’s & Dementia 11 (3), 321–330 .
De Groot, M. , Verhaaren, B.F. , De Boer, R. , Klein, S. , Hofman, A. , van der Lugt, A. , Ikram, M.A. , Niessen, W.J. , Vernooij, M.W. , 2013a. Changes in normal-appearing white matter precede development of white matter lesions. Stroke 44 (4), 1037–1042 . De Groot, M. , Vernooij, M.W. , Klein, S. , Ikram, M.A. , Vos, F.M. , Smith, S.M. , Niessen, W.J. , Andersson, J.L. , 2013b. Improving alignment in tract-based spatial statistics: evalua- tion and optimization of image registration. Neuroimage 76, 400–411 .
Hu, Y. , Modat, M. , Gibson, E. , Ghavami, N. , Bonmati, E. , Moore, C.M. , Emberton, M. , No- ble, J.A. , Barratt, D.C. , Vercauteren, T. , 2018. Label-driven weakly-supervised learn- ing for multimodal deformable image registration. In: Proceedings of the 15th ISBI. IEEE, pp. 1070–1074 .
Ikram, M.A. , Brusselle, G. , Ghanbari, M. , Goedegebure, A. , Ikram, M.K. , Kavousi, M. , Kieboom, B.C. , Klaver, C.C. , de Knegt, R.J. , Luik, A.I. , et al. , 2020. Objectives, de- sign and main findings until 2020 from the rotterdam study. Eur. J. Epidemiol. 1–35 . Ikram, M.A. , van der Lugt, A. , Niessen, W.J. , Krestin, G.P. , Koudstaal, P.J. , Hofman, A. , Breteler, M.M. , Vernooij, M.W. , 2011. The rotterdam scan study: design and update up to 2012. Eur. J. Epidemiol. 26 (10), 811–824 .
Ioffe, S., Szegedy, C., 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167.
Jaderberg, M. , Simonyan, K. , Zisserman, A. , et al. , 2015. Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 .
Keihaninejad, S. , Zhang, H. , Ryan, N.S. , Malone, I.B. , Modat, M. , Cardoso, M.J. , Cash, D.M. , Fox, N.C. , Ourselin, S. , 2013. An unbiased longitudinal analysis frame- work for tracking white matter changes using diffusion tensor imaging with applica- tion to Alzheimer’s disease. Neuroimage 72, 153–163 .
!["Bedingte Urteile in der griechischen Papyri", H. Kupiszewski, "JJP", 9-10, 1956 : [recenzja]](data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)