• Nie Znaleziono Wyników

Bayesian best-worst method

N/A
N/A
Protected

Academic year: 2021

Share "Bayesian best-worst method"

Copied!
10
0
0

Pełen tekst

(1)

Delft University of Technology

Bayesian best-worst method

A probabilistic group decision making model

Mohammadi, Majid; Rezaei, Jafar

DOI

10.1016/j.omega.2019.06.001

Publication date

2019

Document Version

Final published version

Published in

Omega (United Kingdom)

Citation (APA)

Mohammadi, M., & Rezaei, J. (2019). Bayesian best-worst method: A probabilistic group decision making

model. Omega (United Kingdom), 96, [102075]. https://doi.org/10.1016/j.omega.2019.06.001

Important note

To cite this publication, please use the final published version (if applicable).

Please check the document version above.

Copyright

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons. Takedown policy

Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim.

This work is downloaded from Delft University of Technology.

(2)

‘You share, we take care!’ – Taverne project

https://www.openaccess.nl/en/you-share-we-take-care

Otherwise as indicated in the copyright section: the publisher

is the copyright holder of this work and the author uses the

Dutch legislation to make this work public.

(3)

ARTICLE

IN

PRESS

JID:OME [m5G;August31,2019;6:6]

Omega xxx (xxxx) xxx

ContentslistsavailableatScienceDirect

Omega

journalhomepage:www.elsevier.com/locate/omega

Bayesian

best-worst

method:

A

probabilistic

group

decision

making

model

R

Majid

Mohammadi

,

Jafar

Rezaei

Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands

a

r

t

i

c

l

e

i

n

f

o

Article history: Received 10 December 2018 Accepted 3 June 2019 Available online xxx Keywords: Best-worst method Multi-criteria decision-making Bayesian hierarchical model Generalizability

Group decision-making

a

b

s

t

r

a

c

t

The best-worst method (BWM) is a multi-criteria decision-making method which finds the optimal weightsofaset ofcriteriabasedonthepreferencesofonlyone decision-maker(DM)(or evaluator). However,itcannotamalgamatethepreferencesofmultipledecision-makers/evaluatorsintheso-called groupdecision-makingproblem.AtypicalwayofaggregatingthepreferencesofmultipleDMsistouse theaverageoperator,e.g.,arithmeticorgeometricmean.However,averagesaresensitivetooutliersand providerestrictedinformationregardingtheoverallpreferencesofallDMs.Inthispaper,aBayesianBWM isintroducedtofindtheaggregatedfinalweightsofcriteriaforagroupofDMsatonce.Tothisend,the BWMframeworkismeaningfullyviewedfromaprobabilisticangle,andaBayesianhierarchicalmodelis tailoredtocomputetheweightsinthepresenceofagroupofDMs.Wefurtherintroduceanew rank-ingschemefordecisioncriteria,calledcredalranking,where aconfidencelevelisassignedtomeasure theextenttowhichagroupofDMsprefersonecriterionoveroneanother.Aweighteddirectedgraph visualizesthecredalrankingbasedonwhichtheinterrelationofcriteriaandconfidencesaremerely un-derstood. Thenumerical example validatesthe resultsobtained bythe Bayesian BWMwhileit yields muchmoreinformationincomparisontothatoftheoriginalBWM.

© 2019PublishedbyElsevierLtd.

1. Introduction

Multi-criteria decision-making (MCDM) is a sub-discipline of Operations Research, which has growingly gained momentum since its genesis.In atypical MCDM problem, anumber of alter-natives are evaluated basedon ahandful numberofcriteria. The evaluationisusuallyperformedbasedontheelicitationof prefer-encesofadecisionmaker(DM)andcommonlyresultsinsorting, ranking, or selecting the alternative(s). Inorder to do the evalu-ation, we need to find the performance of the alternatives with respect to the criteria, which is called the performance matrix, and the importance (weight) of the criteria. Finding the perfor-mancematrix usually followsa simpleyet crucialdatacollection approach.Weightdeterminationisusuallydonebasedonthe pref-erences of the actual DM. There exist several preference elicita-tionmethodstoinfertheweightsofthedecisioncriteriabasedon the preferences of the DM,including the analytic hierarchy pro-cess (AHP) [1], the analytic network process (ANP) [2], the sim-plemulti-attributeratingtechnique(SMART)[3,4],Swing[5],FARE

R This manuscript was processed by Associate Editor Triantaphyllou. Corresponding author.

E-mail addresses: m.mohammadi@tudelft.nl (M. Mohammadi), J.rezaei@tudelft.nl

(J. Rezaei).

[6],CILOSandIDOCRIW[7],tonamejustafew(see [8]formore MCDMmethods). One ofthe mostrecentlydeveloped preference elicitationmethodsisthebest-worstmethod(BWM)developedby Rezaeiin2015[9,10],whichisapairwisecomparison-basedMCDM method.

WhenwehaveasingleDM,theelicitedpreferencesaredirectly usedinthedecisionanalysiswhileincorporatingtheelicited pref-erencesisnota straightforwardstepwhenthereareseveralDMs. ThelattercaseisusuallycalledgroupMCDM[11–13].Wecan clas-sify group MCDMproblemsinto two categories.Inthe first cate-gory,whichhasanormativeapproach,agroupofDMsseeksa so-lutionwhichsomehowrepresentstheopinionofthewholegroup. Inthesecondcategory,whichisofadescriptiveapproach,wewant tohave a clearunderstanding of thepreferences ofthe DMs. An example of the first category is when a number of DMs from a supply chain management department of a company decides on selectingthe best suppliersforsome materials used in the com-pany[14], whileanexample fromthesecond categoryiswhen a researchertriestounderstandtheimportanceofthecriteriawhich define the quality of passenger transport transit nodes [15]. The mainfocus ofthisstudy ison group MCDM,where wehave the preferencesofagroupofDMs,whetheritisusedforanormative oradescriptiveapproach.

https://doi.org/10.1016/j.omega.2019.06.001

0305-0483/© 2019 Published by Elsevier Ltd.

Please citethis article as:M. Mohammadi and J.Rezaei, Bayesian best-worst method: Aprobabilistic group decision making model, Omega,https://doi.org/10.1016/j.omega.2019.06.001

(4)

2 M. Mohammadi and J. Rezaei / Omega xxx (xxxx) xxx

JID:OME [m5G;August31,2019;6:6]

Fortheweight elicitationmethods thatare basedonthe pair-wise comparison(PC),there are two classesof techniqueswhich can be used to reconcile the discrepancy among DMs [16,17]. Thefirstapproach istheaggregation ofindividualjudgment(AIJ)

[18,19],inwhichthePCs ofdifferentDMsarefirstintegratedinto one,andtheresultingaggregatedPCisthentreatedasasingleDM problemandevaluationis performedaccordingly.Theother class istheaggregationofindividualpriorities(AIP)[20–24].IntheAIP, aweightvectorisfirstcalculatedforeachDM,andtheconsequent weightsarecombinedtoresultinasingleweightvector.Themost popular technique to find the optimal weight forthe AIP is the arithmeticmean[25](forothertechniquesofaggregation,see,for instance[26]).BothAIJandAIPapproachesresultinaweight vec-torwhichrepresentsthepreferencesofthewholegroup.Although botharepractically simple,we losemuchinformationduetothe aggregation.Thatistosay,weusethecentralityfeatureandignore thedispersion property. On top ofthat, averages are sensitive to outliers.Therefore,evenifone decision-makerhasdifferent pref-erencesfrom the entiregroup, he/shewill significantly influence theoverallaggregatedpreferencesofallDMs.

In this study,we propose a novel approach forgroup MCDM. TheproposedapproachisparticularlypresentedfortheBWMdue toitsparticularfeatures. Thepairwise comparisonvectors associ-atedwitheachDMintheBWMcontainintegersonly;hence,they canbe modeled usingthemultinomial distribution. Nevertheless, the proposed approach can be extended for other MCDM meth-odswithsome efforts. Morespecifically,theBayesianBWMis in-troducedwhich can solve the group MCDM problem. The inputs totheBayesian BWMareidenticalto thoseoftheoriginal BWM, whichare the pairwise comparisons.The output is, onthe other hand,theoptimalaggregatedfinalweightsreflectingthetotal pref-erencesofallDMsalongwiththeconfidencelevelforrankingthe criteria.

SincetheBayesianBWMisstochastic,theinputsandoutputsof themethodneedtobemodeledusingprobabilitydistributions.In particular,wemodelthepairwisecomparisonsusingthe multino-mialdistribution,andthefinalaggregatedweightsbytheDirichlet distribution.We further demonstratethat such modeling, though different,is identicalto what is expectedin theMCDM, and the BWMinparticular.

Based on the inputs and required outputs, a Bayesian hierar-chicalmodelisdevelopedto findtheoptimalweights ofallDMs andthe aggregated final weight at once. The proposed model is distinctfromthatoftheAIJinwhichvariousPCMsare combined toreacha consensus matrix.Inthe AIJ,one needs toaccept that someDMscompromiseinordertogetaunanimousranking. How-ever,we merelyviewvarious DMsasstatisticalsamplesbasedon whichthecriteriaareprobabilisticallyevaluated.Thecredalranking

isfurtherintroducedinwhicheachpairofcriteriahasarelation, e.g., < or >,withaconfidencelevel.Theconfidencelevel repre-sentstheextenttowhichonecanbecertainaboutthesuperiority ofacriterion overoneanother. Theconfidencelevel iscomputed basedontheBayesiantestthatisespecially-tailoredbasedonthe proposedhierarchicalmodel.Aweighteddirectedgraphvisualizes theoutcomeofthecredalranking.

The main contributionof thisstudyisto propose a novel ap-proachingroupMCDMandtoapplyBayesianstatisticstoMCDM. ThisapproachisusedfortheBWM,whichisasignificant empow-ermentforthemethodforitsuseinthecontextofgroup decision-making.TheproposedBayesianBWMisparticularlyverypowerful when the goalis to describe the preferences ofa group ofDMs (whocanbetheactualDMs,experts,orusers).

The remainder of this article is structured as follows.

Section2 containstheoriginal best-worstmethod andthe corre-spondingoptimization problem to obtain the optimalweights of thecriteriaforoneDMonly. InSection 3,weprovide the

proba-bilisticinterpretation of theBWMinputs andoutputs andjustify thatsuchaninterpretationwouldpreservetheunderlyingideasin theoriginalBWM.Section4isdedicatedtotheproposedBayesian model,andwepresentthecredalrankinginSection5.The numer-ical exampleregardingtheproposed model isgiveninSection 6, andthearticleisconcludedinSection7.

2. Best-worstmethod

The BWM is a relatively new MCDM method [9,10]. One of the most popular pairwise comparison-based MCDM methods istheAHP[1]whichneedstohavethepairwisecomparisonofall thendecisioncriteriatogether,i.e.,n(n-1)/2pairwisecomparisons. Incontrast,theBWMneeds onlytheso-calledreferencepairwise comparisons,i.e., 2n-3pairwise comparisons.Otherthan this fea-ture of the BWM, which makes it a more data efficient method comparedtoAHP,ithasseveralotherinterestingfeatures.Byfirst selecting the best andthe worst criteriaandthen comparing all the other criteria withthese two criteria, it gives a structure to theproblem.SuchstructurehelpstheDMtoprovidemorereliable pairwisecomparisons [9].Furthermore,theparticularstructureof theBWMleadstotwovectorscontainingonlyintegers,which pre-ventsafundamentaldistanceproblemassociated withtheuseof fractions inpairwise comparisons [27].The original BWMis pre-sentedasanon-linearoptimizationproblem[9],whilethereexists alinearapproximation[10],amultiplicativeversion[28],andsome hybridversionssuchasBWM-MULTIMOORA[29]andBWM-VIKOR. The method has also been extensively used in many real-world applications including, butnot limitedto, transport andlogistics

[30–32], supply chain management [33–39], technology manage-ment[40],riskmanagement[41],scienceandresearchassessment

[42,43],andenergy[44,45](see [46] formorerecentadvancesin theBWManditsapplications).

SincethetwovectorsprovidedbyeachDMintheBWMmight representdifferentcomparisons(withdifferentbestsandworsts), the AIJ is not a proper way of aggregating the preferences of a group of DMs forthis method.Almost all applications presented inexistingliteratureusetheAIPfortheaggregation,i.e.,the arith-meticmeanoftheweightsofthecriteriaobtainedfromthe indi-vidual DMs.There exists anumber ofresearcherswho have pro-poseddifferent waysfor thecaseof group decision-makingwith theBWM[47,48].However, noneofthem hasproposeda wayto find the overall weights of the group in a probabilistic environ-ment.

ThestepsrequiredfortheoriginalBWMareasfollows[9].

Step1: TheDMneedstoprovideasetofdecisioncriteriaC=

{

c1,c2,...,cn

}

.

Step2: TheDMselectsthebest(cB)andtheworst(cW)criteria

fromC.

Inthisstep,theDMonlyselectsthebestandtheworst fromthecriteriasetCidentified inthefirst step.The DMdoesnotconductanypairwisecomparisoninthis stage.BasedontheDM’spreference,thebestcriterion isthemostimportantorthemostdesirablewhilethe worst criterion istheleast importantortheleast de-sirablecriterionamongothers.

Step3: The DM conducts the pairwise comparison between thebest(cB)andtheothercriteriafromC.

In this step,the DM calibrates his/her preferences of the best criterion to the other criteria by a number betweenone andnine, whereone means equally im-portant and nine means extremely more important. Thepairwisecomparisonleadstothe“Best-to-Others”

(5)

M. Mohammadi and J. Rezaei / Omega xxx (xxxx) xxx 3

ARTICLE

IN

PRESS

JID:OME [m5G;August31,2019;6:6]

vectorAB as AB=



aB1,aB2,...,aBn



(1)

whereaBjrepresentsthepreferenceofthebest(cB)to

thecriterioncjC.

Step4: The DM conducts the pairwise comparison between theworst(cW)andtheothercriteriafromC.

Similar to Step 3, the DM needs to calibrate his/her preferences ofthe other criteriaover theworst crite-rionbyanumberbetweenoneandnine.Theresultof thisstepisthe“Others-to-Worst” vectorAWas AW=



a1W,a2W,...,anW



T

(2)

where ajW represents the preference of the criterion cjCovertheworst(cW).

Step5: Obtainingtheoptimalweightsw=

(

w1,w2,...,wn

)

. Given AB and AW, a weight vector w∗ must be

com-puted. The weight vector must be in the neighbor-hood ofthe equationswB/wj=aB j and wj/wW=ajW

for j=1,2,.,n.Thus,onecanminimizethemaximum absolute differences

|

wB

wj − aB j

|

and

|

wj

wW − ajW

|

for all

j=1,2,...,n.Besides,thenon-negativityand sum-to-onepropertyoftheweightvectormustbefulfilled.As a result, the following optimization problemcan find theoptimalweightvectorw∗[9]

min w maxj







wB wj − aB j





,





wj wW − ajW







s.t. n  j=1 wj=1, wj≥ 0

j=1,2,...,n. (3)

Similarly, the weightvector can alsobe calculated by thefollowingproblem[10]

min ξ,w

ξ

s.t.





wB wj − aB j





ξ ∀

j=1,2,...,n





wj wW − ajW





ξ ∀

j=1,2,...,n n  j=1 wj=1, wj≥ 0

j=1,2,...,n. (4)

To check the reliability of the optimal weights, the veracitybetween theinput pairwise comparisonsand their associated weight ratios are checked using the followingconsistencyratio(CR):

CR=

ξ

CI (5)

where

ξ

∗ isthe optimalobjectivevalue ofmodel (4), and CI (consistency index) is a fixed value per aBW,

whichcanbereadfromTable1.

CRisanumberbetween0and1,where0meansfull consistencyandbyincreasingthevalueofCRthe con-sistencyofthepairwisecomparisonsystemis decreas-ing.

3. ProbabilisticinterpretationofBWM

Inthissection,weprovide aprobabilistic interpretationofthe BWMinputsandoutputs,andthenreviewtwoschoolsofthoughts intheprobability estimation,e.g.,frequentist andBayesian,inthe contextoftheBWM.

3.1. Modelinginputsandoutputs:multinomialandDirichlet distributions

Asstatedbefore,thetypicaloutcomeoftheMCDMmethodsis theweightvectorw=[w1,...,wn]suchthat wj≥ 0,nj=1wj=1.

Themagnitude ofeach wj indicates the importanceof the

corre-spondingcriteriacj.

From a probabilistic perspective, the criteria are seen as the random events,andtheir weights are thus their occurrence like-lihoods.Mathematicallyspeaking,suchaninterpretationisinline with the MCDM since wj≥ 0 and nj=1wj=1 according to the probabilitytheoryaswell.Itisfurtheroftheessencetoillustrate that probabilistic modeling makes sense froma decision-making pointofview.

Fortheprobabilisticreasoning,one needs tomodelall the in-putsandtheoutputsastheprobabilitydistributions.First,consider

ABandAWwhicharetheinputstotheBWM.Fromamathematical

pointofview,themultinomial distributioncanmodelthevectors sincealloftheirelementsareintegers.Theprobabilitymass func-tion(PMF)ofthemultinomialdistributionforagivenAw is[49] P

(

AW

|

w

)

=



n j=1ajW



! n j=1ajW! n j=1 wajW j (6)

wherewistheprobabilitydistribution.

Inthemultinomialdistribution,theweight vectoristhe prob-abilitydistribution andAW containsthenumber ofoccurrenceof

eachevent.Apparently,itiscompletelydifferentfromwhatis ex-pected for the BWM represented in Section 2. Interestingly, we showthatmodelingwithmultinomialwouldfulfilltheunderlying ideaoftheBWM.

Based on the multinomial distribution, the probability of the eventj isproportionate tothenumberofoccurrenceoftheevent tothetotalnumberoftrials,i.e.,

wjajW n

i=1aiW

,

j=1,...,n. (7)

Similarly,onecanwritethesameequationfortheworst crite-rionas wWaWW n i=1aiW =n1 i=1aiW (8)

UsingEqs.(7)and(8),oneobtains

wj wW

ajW,

j=1,...,n, (9)

whichisprecisely therelationwe seekintheoriginal BWM pre-sentedinStep5ofSection2.

Similarly, AB can be modeled using the multinomial

distribu-tion.However,AB isdifferentfromAWsincetheformerrepresents

the preferencesof the best over theother criteriawhile the lat-terdenotesthepreferencesoftheothersovertheworst.Thus, AB

Table 1

Consistency Index (CI) Table [9] .

aBW 1 2 3 4 5 6 7 8 9

Consistency Index (CI) 0.00 0.44 1.00 1.63 2.30 3.00 3.73 4.47 5.23

Please citethis article as:M. Mohammadi and J.Rezaei, Bayesian best-worst method: Aprobabilistic group decision making model, Omega,https://doi.org/10.1016/j.omega.2019.06.001

(6)

4 M. Mohammadi and J. Rezaei / Omega xxx (xxxx) xxx

JID:OME [m5G;August31,2019;6:6]

yieldstheinverseoftheweight,i.e.,

AB∼ multinomial

(

1/w

)

(10)

where w is the probability distribution, and / represents the element-wise division operator. Identical to the worst criterion, onecanwrite

1 wjaBj n i=1aBi , w1 BaBB n i=1aBi =n1 i=1aBiwB wjaBj,

j=1,...,n, (11)

whichisagaintheexactrelationweseekintheBWM.

So far, we showed that the multinomial distribution could meaningfullymodeltheinputstotheBWM.Theproblemof find-ingthe weights inthe MCDM problemisthus transferred to the estimationofaprobabilitydistribution.Therefore,onecanusethe statisticalinferencetechniquestofindwinthemultinomial distri-bution.

Aweightvector fortheMCDMmustsatisfythenon-negativity andsum-to-oneproperties.Therefore,an appropriate distribution tomodelthe weightsis tousethe Dirichletdistribution.Givena parameter

α

Rn,theDirichletdistributionoftheweightswis

de-finedas[49] Dir

(

w

|

α

)

= 1 B

(

α

)

n j=1 wαj−1 j . (12)

The distribution has only a vector parameter

α

,and wmeets theconstraintsofanoptimalweightvectorofMCDMsinceitisa probabilitydistribution.

3.2.Estimationoftheweightvector:statisticalinference

Foramoment,assumethatthereisonlyAWintheBWM,then

weconsidertwowidely-acceptedinferencetechniques:frequentist andBayesian. The underlying idea of thefrequentist approach is thatthereisapreciseyetunknownoptimalpoint,andtheeffortis toestimate itbasedontheobservations.Asaresult,theoutcome ofthe frequentistinferenceisa preciseweightvector fora setof criteria.Themaximumlikelihoodestimation(MLE)isarguablythe mostpopularinference techniquewhichfindsthe optimalweight vectorusingthefollowingoptimization

w=arg max

w,n j=1wj=1

P

(

AW

|

w

)

. (13)

Theoptimumof(13)yieldsat

wj=najW i=1aiW

,

j=1,...,n, (14)

whichisindeedthenormalizedAW.Thesamesolutionwillbe

ob-tainedbytheBWMifthe preferencesoftheDMare fully consis-tent.Thus, (14)showsthatthe MLEbearsthesame resultasthe BWMunderspecificcircumstances.

The second approach isthe Bayesianestimation, inwhichthe parameters are approximated by using a distribution ratherthan aprecise pointasis intheMLE.Thus, we firstneed tospecifya priordistributionfortheweightvector.IntheBayesianinference, theDirichletdistribution isusedasthe priorto themultinomial. TheDirichletdistribution can perfectlyrepresentthe weight vec-torsince itsatisfiesbothitsnon-negativityandsum-to-one prop-erties.Using Dirichletasthe prior andmultinomial as the likeli-hood,the posteriordistribution would alsobe Dirichletwiththe posteriorparameter

α

post=

α

+AW.Sincetheprior shouldbe

un-informativesothatitsimpactontheposteriorisminimal,we set thepriorparameter

α

=1.

AsaresultoftheBayesianestimation,thevaluesofwisshown bya Dirichletdistribution.Themode oftheposteriordistribution

μ

Rn withtheparameter

α

post

μ

j=

α

postj− 1 n i=1

α

posti− n = 1+ajW− 1 n i=1

(

aiW+1

)

− n (15) = ajW n i=1aiW ,

j=1,...,n.

Thus,themodeoftheposteriordistributionwouldprovidethe exact MLE.As a result, theBayesian paradigm wouldyield more informationregardingtheeventsunderstudysinceitsoutcomeis a distribution, not apoint. The standard deviation ofsuch a dis-tribution,forinstance,isanindicatorofuncertaintyregardingthe inferenceproblem,whichcanhavedistinctinterpretationswith re-specttotheproblemunderstudy.

So far, we merely considered AW for estimating the weights;

however,itiscriticaltousebothABandAWaccordingtotheBWM.

TheMLEinferencecontainingbothABandAWdoesnotbearan

an-alyticalsolutionduetothecomplexityofthecorresponding opti-mization problem. Further, the simpleDirichlet-multinomial con-jugate cannot encompass the AB and AW together. The problem

compounds when itcomes to havingthe preferences ofmultiple DMs. Considering these issues, a Bayesian hierarchical model is presented in the next section to estimate the optimal weight of thecriteriaconsideringbothAB andAWofmultipleDMs.

4. Bayesianbest-worstmethod

ThissectionpresentsaBayesianhierarchicalmodeltofindthe optimal weights of a set of criteria based on the preferences of multipleDMsusingthebest-worstframework.

4.1. Groupdecision-making:ajointprobabilitydistribution

Assume that the kth DM, k=1,...,K, evaluates the criteria

c1,...,cn by providing the vectors AkB and AkW. We show the set

ofall vectorsofK DMsbyA1:K

B andA1:WK.Fromnowon,the

super-script 1:K would indicate the total of all vectors inthe base. We

alsorepresenttheoveralloptimalweightbywagg.

Theestimationofwagg entailsusingseveralauxiliaryvariables.

Inparticular,wagg iscomputedbasedontheoptimalweightsofK

DMsshownbywk,k=1,...,K.Thus,theproposedBayesianmodel

wouldsimultaneouslycompute wagg andw1:K.Priortoconducting

anystatisticalinference,itisrequiredtowritethejointprobability distributionofallrandomvariablesgiventheavailabledata.Inthe group decision-makingwithin the BWM, A1:K

B andA1:WK aregiven,

andw1:K andwagg must be estimated accordingly. Thus, the

fol-lowingjointprobabilitydistributionissought

P

wagg,w1:K





A1:K B ,A1:WK

. (16)

If the probability in (16) is computed, then the probability of each individual variable can be computedusing the following probabilityrule

P

(

x

)

= y

P

(

x,y

)

(17)

wherexandyaretwoarbitraryrandomvariables.

4.2. Bayesianhierarchicalmodel

Todevelop aBayesianmodel,we firstneed toidentifythe in-dependenceandconditionalindependenceofvariables.Fig.1plots

(7)

M. Mohammadi and J. Rezaei / Omega xxx (xxxx) xxx 5

ARTICLE

IN

PRESS

JID:OME [m5G;August31,2019;6:6]

k

Fig. 1. The probabilistic graphical model of the Bayesian BWM.

the graphical model corresponding to the proposed method. The nodesinthegrapharethevariables.Asaconvention, the rectan-glesaretheobservedvariables,whicharetheinputstothe origi-nalBWM. Thecircularnodesarethevariableswhichmustbe es-timated.Also, arrowsdenotethatthenode inoriginisdependent onthenodeattheotherend.Thatistosay,thevalueofwk is

de-pendentonAk

W andAkB,andthevalueofwaggisalsodependenton wk.

The plate,whichcoversa setofvariables,meansthat the cor-responding variablesare iteratedforeach DM,andwagg is notin

theplatesincethereisonlyonewagg forallDMs.

The conditional independence between various variables is clearbasedonFig.1.Forinstance,Ak

W isindependentofwagggiven wk,i.e., P

Ak W





wagg,wk

=P

Ak W





wk

(18)

Consideringallindependenceamongdifferentvariables, apply-ingtheBayesruletothejointprobability(16)follows

P

wagg,w1:K





A1:K B ,AW1:K

P

A1:K B ,AW1:K





wagg,w1:K

P



wagg,w1:K



=P

(

wagg

)

K k=1 P

AWk





wk

P

AkB





wk

P

wk





wagg

. (19)

where the last equality is obtained using the probability chain rule andthe conditional independenceofdifferent variables, and thefactthateach DMprovideshis/herpreferencesindependently. SincetheestimationoftheparametersinEq.(19)isreliantonthe estimationofothervariables,thereisachainbetweendifferent pa-rameters.The existenceofthechainisthereasonthatthemodel iscalledhierarchical.

Wenowneedtospecifythedistributionsofeachandevery el-ementinEq.(19).WehavealreadyshownthatAB andAWcanbe

perfectlymodeledusingthemultinomialdistributioninthesense thatitpreservestheunderlyingideaoftheBWM.Thereisonlyone differencebetweenAB andAWsince theformershowsthe

prefer-ence of all the criteriaover the worst, while the latter contains thepreferenceofthebestoveralltheothercriteria.Thus,onecan modelthemas

AkB

|

wk∼ multinomial



1/wk



,

k=1,...,K, Ak

W

|

wk∼ multinomial



wk



,

k=1,...,K. (20)

Given wagg, one can expect that each and every wk be in its

proximity. To this end, we reparameterize the Dirichlet distribu-tionwithrespectto itsmeanandaconcentration parameter.The

modelsofwkgivenwaggare

wk

|

wagg∼ Dir

(

γ

× wagg

)

,

k=1,...,K, (21)

wherewaggisthemeanofthedistributionand

γ

isthe

concentra-tionparameter.

Theequationin(21)saysthattheweightvector wk associated

with each DM must be in the proximity of wagg since it is the

meanof the distribution, andtheir closeness isgoverned by the non-negativeparameter

γ

. Such a technique is used in different Bayesian models as well [50]. The concentration parameter also needstobe modeledusinga distribution.Areliableoptionisthe gammadistributionwhichsatisfiesthenon-negativityconstraints, i.e.,

γ

∼ gamma

(

a,b

)

, (22)

wherea andb are the shapeparameters of thegamma distribu-tion.

Wefinallysupplythepriordistributionoverwaggusingan

un-informativeDirichletdistributionwiththeparameter

α

=1as

wagg∼ Dir

(

α

)

. (23)

The specified modeldoes not bear a closed-form solution.As aresult,Markov-chain MonteCarlo(MCMC)techniques[51]must beusedtocomputetheposteriordistribution.FortheMCMC sam-pling,weusethe“justanotherGibbssampler” (JAGS)[52],which isoneofthebestavailableprobabilisticlanguagestodate,to sam-pleandcomputetheposteriordeterminedin(19).Theuseful out-comeofthemodelistheposteriordistributionofweights for ev-erysingleDMandtheaggregatedwagg.

Theproposed Bayesian modelwillreplaceStep 5ofthe origi-nalBWMexplainedinSection2.Infact,theoptimizationproblem issubstitutedwithaprobabilistic modelwhile theinputstoboth methods are identical.However, theproposed modelwould pro-videmoreinformationregardingtheconfidenceoftherelation be-tweeneach pairofcriteria. Theexcessive informationisobtained bydevisinganewBayesiantestbasedontheapproximated distri-butionfromthemodel,whichisexplainedinthenextsection.

5. Credalranking

The modus operandi in the MCDM is to say one criterion is more important than one another merely if its weight, or the weightaverageforthegroupdecision-making,ishigherthanone another. Assume that there are three criteriac1, c2,and c3 with

the weightvector w=[0.49,0.50,0.01]. According tothe MCDM,

c2 is superior to both c1 andc3.However, theconfidence of the

superioritycannotbedeterminedbysolelycomparingtwofigures. Thisisevenmuchmoreimportantwhentheweightvector repre-sentsthepreferencesofagroupofDMs.Todate,therearevarious rankingschemessuchasinterval-basedranking[53],fuzzyranking

[54,55],andrankingbasedongreyrelationalanalysis(GRA)[56]. Thenotionofcredalrankingisnowintroduced,whichcan cali-bratethedegreetowhichonecriterionissuperiortooneanother. Havingtheposteriordistributionofweightswouldhelpgaugethe confidenceoftherelationsbetweenvariouscriteria.Thedifference betweenthe credal ranking andother ranking schemes is that a confidenceiscomputedinthecredalrankingbasedonone distri-bution,i.e., theDirichletdistribution ofwagg,while other ranking

methods usually take two numbers/intervals and try to find the extenttowhichoneissuperior.

Wefirstdefinethecredalordering,whichisthebuilding-block ofcredalranking.

Definition5.1(CredalOrdering). Forapairofcriteriaciandcj,the

credalorderingOisdefinedas

O=

(

ci,cj,R,d

)

(24)

Please citethis article as:M. Mohammadi and J.Rezaei, Bayesian best-worst method: Aprobabilistic group decision making model, Omega,https://doi.org/10.1016/j.omega.2019.06.001

(8)

6 M. Mohammadi and J. Rezaei / Omega xxx (xxxx) xxx

JID:OME [m5G;August31,2019;6:6]

Table 2

Comparison of the original BWM and the Bayesian BWM on the mobile phone selection crite- ria based on the preferences of 50 students.

Basic Physical char. Tech feat. Func Brand Customer BWM 0.1945 0.1623 0.2014 0.2467 0.1277 0.0673 Bayesian BWM 0.1929 0.1776 0.2052 0.2376 0.1277 0.0591

where

R is the relationbetweenthe criteriaciand cj, i.e., <, >,

or=;

d∈[0,1]representstheconfidencesoftherelation.

Definition 5.2 (Credal Ranking). For a set of criteria C=

(

c1,c2,...,cn

)

,thecredalrankingisasetofcredalorderingswhich

includesallpairs(ci,cj),forallci,cjC.

TheconfidenceinthecredalorderingcanprovidetheDMswith moreinformationwhichcan significantlyimprovetheir decisions. Wenowdevise anewBayesian test basedonwhich wecan find theconfidence ofeach credal ordering.The test is predicatedon theposteriordistributionofwagg.Theconfidencethatc

ibeing su-periortocjiscomputedas P

(

ci>cj

)

= I(wagg i >w agg j )P

(

w agg

)

. (25)

where P(wagg) is the posterior distribution of wagg and I is one

ifthe condition in the subscript holds, and zero otherwise. This integrationcanbe approximatedby thesamplesobtainedviathe MCMC.HavingQsamplesfromtheposteriordistribution,the con-fidencecanbecomputedas

P

(

ci>cj

)

= 1 Q Q  q=1 I

(

waggq i >w aggq j

)

P

(

cj>ci

)

= 1 Q Q  q=1 I

(

waggq j >w aggq i

)

(26)

wherewaggq is the qth sample of wagg from the MCMC samples.

Thus, for each pair of criteria, one can compute the confidence that one is superior to one another. The credal ranking can be merely changed into the traditional ranking. In this regard, it is evidentthatP

(

ci>cj

)

+P

(

cj>ci

)

=1.Therefore,ciismore

impor-tantthancjifandonlyifP(ci>cj)>0.5.Asaresult,thetraditional

rankingofcriteriaisobtainable byapplying athresholdof0.5to thecredalranking.

6. Numericalexamples

In this section, a real-world example is analyzed using the BayesianBWM,andthecorrespondingcredalrankingiscomputed andvisualized by usinga weighteddirected graph. The MATLAB implementation of the proposed model can be found at http:// bestworstmethod.com/software/.

Thereal-worldapplicationistheselectionofthemobilephone, towhichtheBWMhasbeenalreadyapplied[9].Theproblem im-plicates the selection of one from a set of mobile phones based ondifferent criteria. Six differentcriteria that studied andfound intheliteratureisusedtoevaluatethemobilephonealternatives. Thecriteriaarebasicrequirement,physicalcharacteristics, techni-calfeatures,functionality,brandchoice,andcustomerexcitement.

The datacollectedin[9]wasfrom50universitystudentswho completelygot familiar withdifferent selected criteriathrough a provideddocument.Variouscharacteristics(e.g.,price,dimension, weight, display) of four particular mobile phones were given to theparticipants.Throughaquestionnaire,studentsfilledinaform

togetthe informationrequiredforthe originalBWM, i.e.,AB and AW.

Thefirstapproach,whichwasemployedin[9],wastofindthe optimal weight vector separately foreach student, and then ag-gregatethemusingthearithmetic meantocompute thefinal ag-gregatedweightvector.ThefirstrowofTable2tabulatesthefinal aggregatedweightsobtainedbytheBWM.Weparticularlyconsider thearithmeticmeantovalidatetheproposedBayesianmodelsince the average of50 participants isa reliable measureaccording to

centrallimittheorem.

The obtained inputs from 50 participants in this experiment werealsogiventotheBayesianBWM,andtheoutcomeofthetest isobtained. Sincetheoutputoftheaggregatedweightisa distri-butionintheBayesianBWM,itisnotpossibletocomparethetwo methodsdirectlyandverifyiftheoutputoftheBayesianBWMis valid.Tohaveameaningfuldiscussionandvalidation,however,we usetheaverage oftheDirichletdistributionofwagg tobe ableto

comparethe twomethods. Thesecond rowofTable 2showsthe averageofthe final aggregatedweight computedbythe Bayesian BWM.

Table 2 indicates that the estimation based on the proposed Bayesian model yields a meaningful result since the average of the estimateddistribution iscentered around the overall average ofeachindividualpreferences.Theexampleshowsthattheoutput of the Bayesian BWM is valid and makes perfect sense. Keep in mindthattheBWMobtainstheweightofeachindividualfirstand then aggregate themby the arithmetic mean,whilethe Bayesian BWMcomputestheaggregateddistributionandalltheindividual preferencesatonceusingprobabilisticmodeling.Theessential dif-ference betweentheBWMandtheBayesianBWMisthatwe can comparethecriteriacolorfully.Thecurrentpracticeistosaya cri-terion is more important than one another ifits average weight hasahighervalue;therefore,itisa blackandwhite(orzero-one) decision.

We compare all pairs of criteria with each other using the credalranking and visualizeits outcome using a weight directed graph.Fig.2displaysthecredalrankingofcriteriaforselectingthe cellphones.Thenodesinthisgrapharethecriteriaandeachedge

Ad BtellsthatcriterionAismoreimportantthanBwiththe con-fidenced.

Based on Fig. 2, functionality is the most important criterion based on the opinions of all participants. At the other extreme, customerexcitement andbrandchoicearetheleastdesirable fea-tures forthe participants in thisexperiment. Further, the degree of certainty about the relationof criteria is alsoevident. For in-stance,technicalfeaturesiscertainlymoreimportantthancustomer excitement,butitismoredesirablethanbasicrequirementwiththe confidenceof0.71.

Asmentionedbefore,thecredalrankingvisualizedinFig.2can be changedintotheconventional rankingmerelyby applyingthe threshold of 0.5 to the obtained confidences. The threshold can vary from one problem to one another, and it is entirely to the DMs’volitiontoopt fora particularthresholdvalue. Forinstance, theconfidence0.71betweentechnicalfeatureandbasicrequirement

might be strongenough for themobile selection problem, butit couldnotberegardedasstrongifthestudywasonanother prob-lem.Inother words,credalranking could beshaped torepresent

(9)

M. Mohammadi and J. Rezaei / Omega xxx (xxxx) xxx 7

ARTICLE

IN

PRESS

JID:OME [m5G;August31,2019;6:6]

Fig. 2. The visualization of the credal ranking for the example of mobile phone selection criteria.

therankingofcriteriaindifferentproblemsbasedontheDMs’ de-siredconfidence.

7. Conclusion

This paperpresents a Bayesian modelfor thegroup decision-makingwithintheBWM.Theproposedmethodmodelstheinputs of theBWMusing themultinomial distribution anditis demon-stratedthatsuchadistributionwouldpreservetheunderlyingidea oftheoriginal BWM.Further, theweightvector ismodeledusing theDirichletdistribution.Theproposed Bayesianmodelisableto compute the weight distribution of each individual in the group decision-making,andanaggregatedfinaldistributionrepresenting theoverallpreferencesofallDMs.Thecredalrankingofcriteriais developedbasedonwhicheach pairofcriteriaareassigneda re-lationandaconfidence.Theconfidencewhichiscomputedbased on aproposed Bayesian modelshowsthe extentto whichone is certain abouttherelationofthecorresponding pairofcriteria.In addition,thecredalrankingisvisualizedusingaweighteddirected graphwhichshowstheinterrelationofcriteriaclearly.

TheproposedBayesianBWMisapromisingmethodinthe con-text ofgroupdecisionmakingwhereone isinterestedinthe col-lectiveopinion ofa group,butatthesametime,one couldcheck therankingoftheweightsinaprobabilisticsense.Thegroupwill bemorecertainabouttherelationoftwocriteriaifitisassociated with a highconfidence level while the relations with low confi-dencelevelshouldbeinterpretedmorecarefully.

There are several avenuesto extend thecurrent research. We aimtoapply such modelingsinother importantMCDMmethods. It isalsointerestingto investigatetheroleofoutliers indifferent groupBWM.Finally,itwouldbeinterestingtoworkonsomeother featuresoftheBayesianBWMsuchasconsistencymeasure.

References

[1] Saaty TL . A scaling method for priorities in hierarchical structures. J Math Psy- chol 1977;15(3):234–81 .

[2] Saaty TL . Decision making for leaders: the analytic hierarchy process for deci- sions in a complex world. RWS publications; 1990 .

[3] Edwards W . How to use multiattribute utility measurement for social deci- sionmaking. IEEE Trans Syst Man Cybern 1977;7(5):326–40 .

[4] Edwards W , Barron FH . Smarts and smarter: improved simple methods for multiattribute utility measurement. Organ Behav Hum Decis Process 1994;60(3):306–25 .

[5] Mustajoki J , Hämäläinen RP , Salo A . Decision support by interval smart/swing-incorporating imprecision in the smart and swing methods. Decis Sci 2005;36(2):317–39 .

[6] Ginevi ˇcius R . A new determining method for the criteria weights in multicri- teria evaluation. Int J Inf TechnolDecis Making 2011;10(06):1067–95 .

[7] Zavadskas EK , Podvezko V . Integrated determination of objective criteria weights in mcdm. Int J Inf TechnolDecis Making 2016;15(02):267–83 .

[8] Triantaphyllou E . Multi-criteria decision making methods. In: Multi-criteria de- cision making methods: A comparative study. Springer, Boston, MA; 20 0 0. p. 5–21 .

[9] Rezaei J . Best-worst multi-criteria decision-making method. Omega 2015;53:49–57 .

[10] Rezaei J . Best-worst multi-criteria decision-making method: some properties and a linear model. Omega 2016;64:126–30 .

[11] Davey A , Olson D . Multiple criteria decision making models in group decision support. Group Decis Negotiat 1998;7(1):55–75 .

[12] Wu Z , Xu J . Managing consistency and consensus in group decision making with hesitant fuzzy linguistic preference relations. Omega 2016;65:28–40 .

[13] Yan H-B , Ma T , Huynh V-N . On qualitative multi-attribute group decision making and its consensus measure: a probability based perspective. Omega 2017;70:94–117 .

[14] Sanayei A , Mousavi SF , Yazdankhah A . Group decision making process for supplier selection with vikor under fuzzy environment. Expert Syst Appl 2010;37(1):24–30 .

[15] Groenendijk L , Rezaei J , Correia G . Incorporating the travellers’ experience value in assessing the quality of transit nodes: a rotterdam case study. Case Stud Transp Policy 2018;6(4):564–76 .

[16] Forman E , Peniwati K . Aggregating individual judgments and priorities with the analytic hierarchy process. Eur J Oper Res 1998;108(1):165–9 .

[17] Forman E , Peniwati K . Aggregating individual judgments and priorities with the analytic hierarchy process. Eur J Oper Res 1998;108(1):165–9 .

[18] Blagojevic B , Srdjevic B , Srdjevic Z , Zoranovic T . Heuristic aggregation of in- dividual judgments in ahp group decision making using simulated annealing algorithm. Inf Sci 2016;330:260–73 .

[19] Forman E , Peniwati K . Aggregating individual judgments and priorities with the analytic hierarchy process. Eur J Oper Res 1998;108(1):165–9 .

[20] Abel E , Mikhailov L , Keane J . Group aggregation of pairwise comparisons using multi-objective optimization. Inf Sci 2015;322:257–75 .

[21] Herrera-Viedma E , Herrera F , Chiclana F . A consensus model for multiperson decision making with different preference structures. IEEE Trans Syst ManCy- bern-Part A 2002;32(3):394–402 .

[22] Morais DC , de Almeida AT . Group decision making on water resources based on analysis of individual rankings. Omega 2012;40(1):42–52 .

[23] Ramanathan R , Ganesh L . Group preference aggregation methods employed in ahp: an evaluation and an intrinsic process for deriving members’ weightages. Eur J Oper Res 1994;79(2):249–65 .

[24] Vinogradova I , Podvezko V , Zavadskas E . The recalculation of the weights of criteria in mcdm methods using the bayes approach. Symmetry 2018;10(6):205 .

[25] Ishizaka A , Labib A . Review of the main developments in the analytic hierarchy process. Expert Syst Appl 2011;38(11):14336–45 .

[26] Grošelj P , Stirn LZ , Ayrilmis N , Kuzman MK . Comparison of some aggre- gation techniques using group analytic hierarchy process. Expert Syst Appl 2015;42(4):2198–204 .

[27] Salo AA , Hämäläinen RP . On the measurement of preferences in the analytic hierarchy process. J Multi-Criteria Decis Anal 1997;6(6):309–19 .

[28] Brunelli M , Rezaei J . A multiplicative best-worst method for multi-criteria de- cision making. Oper Res Lett 2019;47(1):12–15 .

[29] Hafezalkotob A , Hafezalkotob A , Liao H , Herrera F . Interval multimoora method integrating interval borda rule and interval best-worst-method-based weight- ing model: case study on hybrid vehicle engine selection. IEEE Trans Cybern 2019 .

[30] Rezaei J , van Wulfften Palthe L , Tavasszy L , Wiegmans B , van der Laan F . Port performance measurement in the context of port choice: an mcda approach. Manag Decis 2018;57(2):396–417 .

[31] Groenendijk L , Rezaei J , Correia G . Incorporating the travellers’ experience value in assessing the quality of transit nodes: a rotterdam case study. Case Stud Transp Policy 2018;6(4):564–76 .

Please citethis article as:M. Mohammadi and J.Rezaei, Bayesian best-worst method: Aprobabilistic group decision making model, Omega,https://doi.org/10.1016/j.omega.2019.06.001

(10)

8 M. Mohammadi and J. Rezaei / Omega xxx (xxxx) xxx

JID:OME [m5G;August31,2019;6:6]

[32] Rezaei J , Hemmes A , Tavasszy L . Multi-criteria decision-making for complex bundling configurations in surface transportation of air freight. J Air Transp Manag 2017;61:95–105 .

[33] Gupta H , Barua MK . A novel hybrid multi-criteria method for supplier se- lection among smes on the basis of innovation ability. Int J Logist ResAppl 2018;21(3):201–23 .

[34] Kusi-Sarpong S , Gupta H , Sarkis J . A supply chain sustainability innovation framework and evaluation methodology. Int J Prod Res 2019;57(7):1990–2008 .

[35] Vahidi F , Torabi SA , Ramezankhani M . Sustainable supplier selection and order allocation under operational and disruption risks. J Clean Prod 2018;174:1351–65 .

[36] Ahmad WNKW , Rezaei J , Sadaghiani S , Tavasszy LA . Evaluation of the external forces affecting the sustainability of oil and gas supply chain using best worst method. J Clean Prod 2017;153:242–52 .

[37] Ahmadi HB , Kusi-Sarpong S , Rezaei J . Assessing the social sustainability of supply chains using best worst method. Resour Conserv Recycl 2017;126:99–106 .

[38] Rezaei J , Nispeling T , Sarkis J , Tavasszy L . A supplier selection life cycle ap- proach integrating traditional and environmental criteria using the best worst method. J Clean Prod 2016;135:577–88 .

[39] Rezaei J , Wang J , Tavasszy L . Linking supplier development to supplier segmen- tation using best worst method. Expert Syst Appl 2015;42(23):9152–64 .

[40] Gupta H , Barua MK . Identifying enablers of technological innovation for indian msmes using best–worst multi criteria decision making method. Technol Fore- cast Soc Change 2016;107:69–79 .

[41] Torabi SA , Giahi R , Sahebjamnia N . An enhanced risk assessment framework for business continuity management systems. Saf Sci 2016;89:201–18 .

[42] Salimi N . Quality assessment of scientific outputs using the bwm. Scientomet- rics 2017;112(1):195–213 .

[43] Salimi N , Rezaei J . Measuring efficiency of university-industry ph. d. projects using best worst method. Scientometrics 2016;109(3):1911–38 .

[44] Ren J . Multi-criteria decision making for the prioritization of energy systems under uncertainties after life cycle sustainability assessment. Sustain Prod Con- sump 2018;16:45–57 .

[45] Gupta H . Evaluating service quality of airline industry using hybrid best worst method and vikor. J Air Transp Manag 2018;68:35–47 .

[46] Mi X , Tang M , Liao H , Shen W , Lev B . The state-of-the-art survey on inte- grations and applications of the best worst method in decision making: why, what, what for and what’s next? Omega 2019 .

[47] Hafezalkotob A , Hafezalkotob A . A novel approach for combination of individ- ual and group decisions based on fuzzy best-worst method. Appl Soft Comput 2017;59:316–25 .

[48] Mou Q , Xu Z , Liao H . An intuitionistic fuzzy multiplicative best-worst method for multi-criteria group decision making. Inf Sci 2016;374:224–39 .

[49] Forbes C , Evans M , Hastings N , Peacock B . Statistical distributions. John Wiley & Sons; 2011 .

[50] Kruschke J . Doing Bayesian data analysis: a tutorial with R, JAGS, and Stan. Academic Press; 2014 .

[51] Gilks WR , Richardson S , Spiegelhalter D . Markov chain Monte Carlo in practice. CRC press; 1995 .

[52] Plummer M. Jags: just another gibbs sampler. 2004.

[53] Moore RE , Kearfott RB , Cloud MJ . Introduction to interval analysis, 110. Siam; 2009 .

[54] Yatsalo BI , Martínez L . Fuzzy rank acceptability analysis: a confidence measure of ranking fuzzy numbers. IEEE Trans Fuzzy Syst 2018;26(6):3579–93 .

[55] Dubois D , Prade H . Ranking fuzzy numbers in the setting of possibility theory. Inf Sci 1983;30(3):183–224 .

[56] Xie N-m , Liu S-f . Novel methods on comparing grey numbers. Appl Math Model 2010;34(2):415–23 .

Cytaty

Powiązane dokumenty

[r]

Optrekkend vocht uit grondwater komt meestal samen voor met de aanwezigheid van hygroscopische zouten: als dit het geval is, zullen de MC- en HMC-lijnen elkaar kruisen op de

These visible plumes are the visual evidence of evaporation processes happening dur- ing rain events, where the splash-droplet evaporation pro- cess provides the required water vapor

Publikacja druga (oznaczona num erem 3) zawiera informacje biograficzne do­ tyczące 170 osób: farm aceutów i osób im bliskich oraz osób związanych z farm

In [12] a control algo- rithm is developed based on the dynamic inversion of a nonlinear vehicle model and force allocation using nonlinear optimization with optimization

plied to the study of single bubbles formed by boiling, and showed that vapor bub- bles with sufficiently small gas content would collapse without &#34;rebound&#34;... In 1955 C 3)

schematic diagram oi tl.e nearstoe circulation systei... 528

Skutnabb-Kangas (2000: 502) maintains that a universal covenant of Linguistic Human Rights should guarantee, among other things, that “any change of mother tongue is