Secure or usable computers? Revealing employees’ perceptions and trade-offs by means of a discrete choice experiment

(1)

Secure or usable computers? Revealing employees’ perceptions and trade-offs by means

of a discrete choice experiment

Molin, Eric; Meeuwisse, Kirsten; Pieters, Wolter; Chorus, Caspar

DOI

10.1016/j.cose.2018.03.003

Publication date

2018

Document Version

Final published version

Published in

Computers and Security

Citation (APA)

Molin, E., Meeuwisse, K., Pieters, W., & Chorus, C. (2018). Secure or usable computers? Revealing

employees’ perceptions and trade-offs by means of a discrete choice experiment. Computers and Security,

77, 65-78. https://doi.org/10.1016/j.cose.2018.03.003

Important note

To cite this publication, please use the final published version (if applicable).

Please check the document version above.

Copyright

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons. Takedown policy

Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim.

This work is downloaded from Delft University of Technology.

(2)

‘You share, we take care!’ – Taverne project

https://www.openaccess.nl/en/you-share-we-take-care

Otherwise as indicated in the copyright section: the publisher

is the copyright holder of this work and the author uses the

Dutch legislation to make this work public.

(3)

Available

online

at

www.sciencedirect.com

journalhomepage: www.elsevier.com/locate/cose

Secure

or

usable

computers?

Revealing

employees’

perceptions

and

trade-offs

by

means

of

a

discrete

choice

experiment

Eric

Molin

a,∗

_,

_Kirsten

_Meeuwisse

b

_,

_Wolter

_Pieters

c

_,

_Caspar

_Chorus

d

a_Engineering_Systems_and_Services_Department,_Faculty_of_Technology,_Policy_and_Management,_Delft_University_of Technology,Delft,TheNetherlands

b_Cyber_Security,_Deloitte,_Amsterdam,_The_Netherlands

c_Values,_Technology_and_Innovation_Department,_Faculty_of_Technology,_Policy_and_Management,_Delft_University_of Technology,Delft,TheNetherlands

d_Engineering_Systems_and_Services_Department,_Faculty_of_Technology,_Policy_and_Management,_Delft_University_of Technolog,Delft,TheNetherlands

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received28May2017 Revised16February2018 Accepted18March2018 Availableonline6April2018

Keywords:

Informationsecurity Securitymeasures Securityperception Usabilityperception Discretechoiceexperiments Discretechoicemodels Employees’preferences

a

b

s

t

r

a

c

t

Itisoftensuggestedintheliteraturethatemployeesregardtechnicalsecuritymeasures (TSMs)asuser-unfriendly,indicatingatrade-offbetweensecurityandusability.However, thereislittleempiricalevidenceofsuchatrade-off,noraboutthestrengthofthe asso-ciatednegativecorrelationandtheimportanceemployeesattachtobothproperties.This paperintendstofilltheseknowledgegapsbystudyingemployees’trade-offsconcerning theusabilityandsecurityofTSMswithinadiscretechoiceexperiment(DCE)framework. InourDCE,employeesareaskedtoindicatethemostpreferredsecuritypackagesthat de-scribecombinationsofTSMs.Inaddition,securityandusabilityperceptionsofthesecurity packagesareexplicitlymeasuredandmodelled.Themodelsestimatedfromtheseobserved responsesindicatehoweachTSMaffectsperceivedsecurity,perceivedusabilityand prefer-ence.Thepaperfurtherillustrateshowthemodellingresultscanbeappliedtodesignhighly securepackagesthatarestillpreferredbyemployees.Thepaperalsomakesa methodolog-icalcontributiontotheliteraturebyintroducingdiscretechoiceexperimentstothefieldof informationsecurity.

1. Introduction

Morethan40millioncybersecurityincidentsarereported ev-eryyear,andthedamagedonebycybercrimetotheprivate sectorisestimatedtoamounttohundredsofbillionsofeuros every year(ISACAandRSAConference,2015;Gandal,2015). These numbers indicate thatinformationsecurity isof ut-most importanceforcompanies. Companies protect

them-∗_{Corresponding}_author.

E-mailaddress:e.j.e.molin@tudelft.nl(E.Molin).

selvesfromdatabreachesandcyberattacksbyimplementing arangeoftechnicalsecuritymeasures(TSMs).Ifemployees usethesemeasuresasintended,morestringentsecurity mea-sureswouldbydesignresultinhigherlevelsofsecurity, al-thoughtheymayhaveanegativeimpactonproductivity. How-ever,ifemployeesperceivethosemeasuresaslessusablethey mayfindwaystocircumventthem,whichpotentiallymakes themlessorevencounter-effective(Dinevetal.,2006; Kirlap-posetal.,2015;PostandKagan,2007).Forexample,if employ-eesareforcedtochangetheirpasswordeveryweek,theymay writedowntheirpasswordsonpost-itsattachedtotheirdesk. Althoughitisusuallythecompanies’ChiefInformation

Secu-https://doi.org/10.1016/j.cose.2018.03.003

(4)

rityOfficer(CISO)whomakesthedecisionsontechnical secu-ritymeasures,itisthecompliancebehaviouroftheemployees thatlargelydeterminestheresultinglevelofthecompany’s cyber-orinformationsecurity.

CISO’sthushavetomakecomplicateddecisions, involv-ingnotonlysecurity,butalsocost(limitedbudget),usability, andimpactonproductivity,andthesuccessoftheirdecisions partlydependsonthepreferencesandbehaviourofthe em-ployees.Itisoftensuggested(seeliteraturereviewinthenext section)thatthemostsecuremeasuresareperceivedby em-ployeesasparticularlyuser-unfriendly,suggestingthatCISO’s havetomakeatrade-offinthisregard.Butthereisinfactlittle empiricalevidenceaboutwhethersuchatrade-offexists,nor aboutthestrengthofthiscorrelation.Furthermore,itis un-knownwhatimportanceemployeesattachto(perceived) se-curityand(perceived)usabilityofinformationsecurity mea-sures.ThismakesithardforCISOstoselectthosetechnical measuresthatprovideahighlevelofsecuritybutstillare con-sideredsufficientlyusable,enablingeffectivesecurity deploy-ment.Therefore,itisimportanttostudytheemployees’ be-haviour,inparticularinrelationtothesupposedtrade-off be-tweenthesecurityofsuchmeasuresandtheirusability.This canbedonewithintheframeworkofdiscretechoicetheory (DCT)anddiscretechoiceexperiments(DCE),whichis partic-ularlysuitabletostudytrade-offs.Tothisbestofour knowl-edge,thismethodofdatacollection(DCE)andanalysis(DCT) hasnotbeenusedbeforeinthecontextofcyber-or informa-tionsecurity.

Thispaperintendstofilltheabovedescribedknowledge gapbyempiricallystudyingemployees’trade-offsconcerning the usabilityandsecurityofinformationsecuritymeasures within aDCE framework.In our DCE employeesare asked to provideresponses tohypotheticalsecurity packages de-scribingcombinationsoftechnicalsecuritymeasures.Our ap-proachismoresophisticatedthantheusualexperimentalset upusedforchoiceanalysis,inthesensethat– inadditionto observingchoicesamongthosesecuritypackages– wealso explicitlymeasureandmodelperceptionsconcerningthe se-curitypackagesintermsofsecurityandusability.Dataare col-lectedusinganon-lineexperimentwhichwascompletedbya sampleof230employees.Theinsightstheapplicationofthis methodologyrevealscanbeusedbysystemadministratorsto choosesecuritymeasuresthatareperceivedtobeusableand mayincreasecompliancebehaviour.

The next section discusses related work; afterthat, we provide aconceptualframework andderive research ques-tions.Subsequently,theconstructionoftheexperiment,the datacollectionandthemodelestimationproceduresare ex-plained.Thisisfollowedbyapresentationanddiscussionof theresultsoftheestimatedmodels,includingimplicationsfor practice.Finally,theresultsarediscussedinlightofthe liter-atureandavenuesforfurtherresearcharediscussed.

2. work

Informationsecurityresearchstartedoutwithdevising tech-nicalsolutionstoprotectinformation.Suchsolutionswould notalwaystakeusabilityintoaccount.Instead,themainfocus wasonmakingthetechnology“work”,andonmakingusers

complywiththetechnology-imposedusagerequirements.In asense,therewasanadversarialrelationwiththeuser,who hadtobe“changed” inordertofitwiththetechnological de-sign.Inaseminalpaper,AdamsandSasse(1999)pointedout that“usersarenottheenemy”:designswouldneedtotakethe userexperienceintoaccount(user-centreddesign)inorderto beeffective.

Still,therelationbetweensecurityandusabilityremained unclear. Schultz(2007)alreadystatedthat“although numer-ousauthorshavearguedfortheneedtopaymoreattention tousabilityconsiderationsininformationsecurity,relatively fewpaperspresentempiricalresultsontherelationship be-tweenusabilityandinformationsecurity.” Itisoftenclaimed thatsecurityandusabilityaretwoconflictinggoals:improving onewillnegativelyaffecttheother(Andersson,2013;Kainda etal.,2010;Nurseetal.,2011).Theassumedrelationisa neg-ativecorrelation:ifsecuritygoesup,usabilitygoesdownand ifusabilitygoesupsecuritygoesdown.Consideracomputer withoutpasswordprotection.Itisclearlyusable,butnot se-cure.Ontheotherhand,acomputeronwhichyouhaveto au-thenticateyourselfeveryfiveminutesbyprovidingyour pass-wordcouldbeverysecure,butnotuser-friendlyatall;users arelikelytobeunwillingtousesuchacomputer(Cranorand Garfinkel,2004).

Herley (2009) analysed the motivation of employees to complywithsecuritymeasuresintermsofcostsandbenefits, anotionwhichismorebroadlysupportedbythewell-known Technology AcceptanceModel (Davis, 1986; Venkateshand Davis,2000).Hearguesthatemployees’perceptionofthe ben-efitsassociatedwith(complyingwith)acybersecurity mea-suredependsontheextenttowhichtheyperceiveitto actu-allycontributetosecurity.Hedefinesperceivedcostsinterms oftheeffortittakesemployeestocomply:themoreeffortit takes,thelessameasureisperceivedtobeuserfriendlyor ‘usable’.Similarly, Beautementetal.(2009)describeamodel inwhichemployeesmakeacost-benefitanalysisinrelation tothe(non-)encryptionofUSBsticksfordatatransfer,and as-sociatedconfidentialityandavailabilityrisks.Buttheideaof ageneraltrade-offbetweensecurityandusabilityisdisputed. Forexample,Caputoetal.(2016)usethreecasestudies show-ingthatatrade-offdoesnotalwaysexist.

Inanycase,thereisaconsensusontheneedtoconsider usabilitywhendesigningsecuritysolutions.Inthislineof re-search,manypapershavearguedfordifferentapproachesto takingusabilityintoaccountinthedesignofsecurity tech-nology.Insuchapproaches,thefocusisonthedesign,thus whatisrequiredofdesignmethodsinordertoleadtousable designs.GutmannandGrigg(2005)discusseddifferent possi-bleoptionsforhowthetwocanbecombinedinthedesign process.Dhillonetal.(2016)usedvalue-basedobjectivesasa meanstosupportdecisionsonbalancingsecurityand usabil-ity,whereasMohamedetal.(2016)focusedonmentalmodels.

Furnell(2016)concludedthatusabilityhasreceivedmore at-tentionovertheyears,andthatmorechoicesbetween secu-ritymechanisms(withdifferentlevelsofperceivedusability fortheindividualuser)areavailabletousers.

Tothe extent that usability hasbeen evaluated empiri-cally,thismostlyconcernedtheuser-friendlinessofasingle

securitytechnology,asawaytopoint outproblemsin cur-rentapproaches,orameansofvalidationofabetterdesign

(5)

Fig.1– Conceptualframework.

(cf. Brostoffetal.,2010;CatuognoandGaldi,2014;Shenget al.,2006).Thisdoesnotrevealthetrade-offsthatusersmake whenhavingtheopportunitytochoosebetweendifferent de-signswithdifferentusabilityandsecuritylevels.Thelatteris particularlyrelevantinthecontextofconcernsthat employ-eesmaybypasscompanytechnologyandusealternative(free butcommercial)servicesinstead,possiblywiththeirown se-curityadd-ons– anotionwhichhasbeencalled“shadow se-curity” (Kirlapposetal.,2015).

3. Conceptual

framework

and

research

questions

Inordertounderstandhowusersmakechoicesbetween dif-ferentproducts/serviceswithdifferentusabilityandsecurity characteristics,weneedtoinvestigatetheirpreferencesinthe faceofsuchdifferentconfigurations.Asdiscussedearlier,the company’sCISOdecidesuponthesecuritytechnology;hence, employeestypicallycannotfreelychoosethesecurity pack-ages oftheir preference.However,in this study weaim to studytheprocessasiftheycould,soweareinterestedinthe choicestheymakeiftheycouldfreelychoosetheirsecurity packagesandhowtheyarriveatthischoice;weassumethat morepreferredsystemswillresultinhighercompliance be-haviour.

Tostudypreferences,weleveragetheparadigmofDiscrete Choice Theory(e.g., Ben-Akivaand Lerman, 1985) and Dis-creteChoiceExperiments(Louviereetal.,2000;Hensheretal., 2005).Thesearequantitativeapproachesthatareusually em-ployedincombinationwiththeaimtoempiricallyelicitthe weights of different attributes inthe preferencesof users. Morespecifically,weconductourresearchwithintherandom utility framework (e.g. Manski, 1977;McFadden, 2001). This frameworkassumesthatpeoplechoosethatalternativefrom asetofavailableoptionsfromwhichtheyderivethehighest utility;andthatpartofutilitythatcanberelatedto observ-ablefactors(suchastheattributesofalternatives)while an-otherpartisrandom,fromtheviewpointoftheanalyst.The approachrequirestheobservationofchoicesamong alterna-tivesthataredescribedinseveralattributes.Inthisstudy,the

attributesaretechnicalsecuritymeasures(TSMs)thatcanbe takenbycompaniestoprotectinformationstoredat comput-ers.Thealternativesdescribecombinationsofattributesand thusrepresentpackagesofTSMs.Theconstructionofthese alternativesisdiscussedinthenextsectioninfulldetail.

Theseconstructedalternativesalsoallowustostudythe trade-offbetweensecurityandusability.Atfirstsight,sucha studywouldrequireestablishingsecurityandusabilitylevels fordifferentsecuritydesigns.However,becauseitis compli-catedtoassesssecurityobjectively(cf.Sanders,2014)and be-causeemployeesmakechoicesamongalternativesbasedon theirperceptionofthealternatives(asopposedontheir ob-jectivecharacteristics),wewillexplicitlymeasurethese per-ceptions.

Fig.1summarizes the conceptual framework underlying ourstudy.Aswillbeexplainedinthefollowingsection, se-curitypackagesareconstructedthatconsistofdifferent com-binationsofTSMs.Thesepackagesareevaluatedby employ-ees in terms of perceived security and perceived usability. Thecorrelationbetweentheseobservedevaluationsindicates whetherthisrelationshipisnegativeashasbeensuggested intheliterature.Fromthe observedperceptionevaluations, modelsareestimatedthatindicatetowhatextenteachofthe TSMsaffectsperceivedsecurityandperceivedusabilityofa securitypackage.Furthermore,choicesareobservedbetween differentsecuritypackages.Fromtheseobservedchoices,a choicemodel is estimated that indicates whether security orusability hasastronger effectonutility and assuchon choices.Moreover,itisexaminedwhethertheeffectofTSMs onchoices(utility)isfullyorpartiallymediatedbysecurity andusability perceptions.Thesolidlines representthefull mediationoftheeffectoftheTSM’sonutilitybythetwo per-ceptionvariables;thedashedlinerepresentsthedirecteffect oftheTSM’sonutility.

Tosummarize,thispaperaimstoanswertothefollowing researchquestions:

• Do perceived usability and perceived security correlate negativelyassuggestedintheliterature?

• Are morerestrictivemeasuresperceived asmoresecure andaslessuser-friendly?

(6)

Table1– Selectedattributes(TSMs)andtheirlevels.

Attribute Level1 Level2 Level3

Passwordlength Norestrictions Minimal8characters Minimal 8 characters,1uppercase letter,1 special characterand1 numericcharacter

Passwordexpiryfrequency Never Onceayear Onceaquarter

Browserrestrictions Everybrowserisallowed Obligatorybrowser Filesharinginsidecompany Norestrictions Viacorporateshareddrive E-mailtosomeoneoutside

thecompany

Norestrictions Warningmessagewithe-mail Pop-upmessagewithe-mail whichcontainsconfidentialword

• Does perceived usability or perceived security weigh strongerinemployees’preferencesforcomputersecurity? • Arealleffectsoftechnicalsecuritymeasuresonchoice

me-diatedbyperceivedusabilityandperceivedsecurity?

Nexttotheseempiricalquestions,thisstudyalsoaimsto introducethechoicemodellingparadigmintheinformation securitycommunityandevaluatepossibilitiesforfurther re-searchalongtheselines.

4. Methodology

In thissection,the discretechoiceexperiment (DCE)is ex-plained.First,wefocusonhowthetechnicalsecurity mea-suresareselected.Thisisfollowedbyadescriptionoftheway thesearecombinedtoarriveatchoicealternatives.Next,the measurementtasksareexplained.Andfinally,themodel esti-mationanddatacollectionproceduresarediscussed.For rea-sonsofspacelimitations,weareunabletocoverallnuances andsubtletiesthatplayaroleindesigningDCEs.Interested readersarereferredtoHensheretal.(2005)forafull descrip-tion.

4.1. Technicalsecuritymeasures(TSMs)

The alternatives presented to participants in the Discrete ChoiceExperimentinvolvecombinationsofTSMs.Wedefine aTSMasanelectronicsecuritymethodthatprotects infor-mationonanofficecomputer.Hence,thisdefinitionexcludes measuresthatcannotbeappliedonacomputeraswell as measuresthatemployeesmaytakeathometoprotecttheir computer.

ToarriveatalistofTSMstobeincludedintheexperiment, alonglistofdifferentkindsofTSMsmentionedinthe liter-aturewasmade(e.g., Nurse,etal.,2011;Hagenet al.,2008; Kaindaetal.,2010).Themeasuresthatdidnotfitthe defini-tion wereremovedfrom thelistandthemeasuresthatare fairlysimilar toeach other were grouped together.The re-sultinglistwasdiscussedwithtwoexpertsofamajor con-sultantinthefieldofinformationsecurity,whohaveample ofexperiencewithadvisingvariousclientsinthatregard. Fi-nally,themostcommonlyusedTSMswereselectedtoensure thatthesearefamiliartoallrespondents.Theresultingseven attributes were tested ina pilot research, afterwhich two moreattributeswereexcludedbecausetheircontentpartially

overlappedandrespondentsreportedhavingtroubles under-standingtheirmeaning.Theresultingfiveattributes(TSMs) andtheirlevels,whichrepresentthedifferentvaluesthe at-tributescantakeinourexperiment,arelistedinTable1.

4.2. Constructionofalternatives

Toarriveatalternatives from whichparticipantsare asked to choose during the DCE, the attribute levels (TSM-specifications) are combined according toan experimental design.Becauserespondents,inaddition tobeing askedto chooseamongalternatives(securitypackageswhichconsist ofcombinationsofTSMs),arealsoaskedtoexplicitlyevaluate eachalternativeintermsofitsperceivedusabilityand secu-rity,thetotalnumberofalternativestobeconstructedhadto belimited.Thiswayweavoidconstructingameasurement taskthatistoodemandingforrespondents,whichmight trig-gerworkoverloadandrespondentfatigue,whichinturncould leadtounreliableresponses.Withthisconstraintinmind,we constructedchoicesets ofthreealternatives each,knowing thatachoicefromasetofthreealternativesprovidesmore preference-information thana choicefrom aset withonly twoalternatives.Thisapproachreducesthenumberofchoice taskshavingtobeattendedtobyparticipants.

Constructingalimitednumberofchoicesetswhilestill be-ingabletoestimatereliableparameters,canbeaccomplished bybasingtheconstructionofthechoicesalternativesonan efficientexperimental design(e.g. Rose and Bliemer, 2009). Efficientexperimentaldesignsmaximize informationabout thepreferencesandtrade-offsobtainedfromeachobserved choiceobservation.Thisisachievedby,forexample, avoid-ing achoicetaskwhich containsan alternativethat domi-natesallotherchoicealternatives(i.e.outperformsitonevery attribute).Moregenerally,efficientexperimentaldesigns in-volvebalancingtheutilitiesofthealternativesineachchoice set.Tocreatesuchabalance, insightisneededinthe util-itiesofalternatives,whichrequirespriorparametervalues, thatis,thebestestimatesoftherealparametervaluesbythe analyst.

Asthisisthefirststatedpreferenceexperimentconducted onthistopic,nopriorparametervalueswereavailablefrom previous research.Therefore,a pilotresearchisconducted. Thechoicesetsforthispilotstudy,eachconsistingthree al-ternatives,areconstructedfromaso-callednearorthogonal designresultingin8choicesets.Thispilotexperimentwas filledoutby31respondentsrecruitedfromthepersonal

(7)

net-Fig.2– Exampleoftheperceptionratingtask.

work of one of the authors. A multinomial logit model is estimatedfromtheseobservedchoicesandtheestimated pa-rameterswereselectedasthepriorsforconstructingthe ef-ficientdesignforthemainexperiment.Thisresultedinthe constructionof6choicesetsofthreealternativeseach,inthe finaldesign.

4.3. Measurementtask

Withrespecttoeachofthechoicesets,werequested respon-dentstoperformtwodifferenttasks.First,theywereasked toevaluateeachalternative(securitypackage)intermsof us-abilityandsecurity.Tothateffect,eachsinglealternative is shownontheparticipant’scomputerscreen,onebyone.The respondentthenevaluatedthesecurityand usabilityofthe alternativebymeansoffive-pointratingscales,runningfrom (1)highlyinsecureto(5)highlysecureandfrom(1)very user-unfriendlyto(5)veryuser-friendly,respectively.After provid-ingtheresponsestopackageA,thesecondalternative, pack-ageB,ofthesamechoicesetappearsonthecomputerscreen. Afterprovidingtheratingsforthispackage,thethirdandfinal packageCofachoicesetisshown.Fig.2presentsa screen-shot of the perception rating task.Note that package B is placedinthemiddle.Thislocationcorrespondswiththe lo-cationofthatalternativeinthechoicetask(seeFig.3),which isdiscussednext.

Afterallthethreealternativesareratedonebyoneinterms ofperceivedsecurityandperceivedusability,theentirechoice

set ispresentedon the screen,which thus consistsof the samethreepackagesthe respondentsratedjustbefore. Re-spondentsarethenrequestedtoindicatewhichofthethree packagestheywouldpreferatwork.Anexampleofthischoice taskispresentedinFig.3.Notethattheperceivedsecurityand perceivedusability ratingswhichthe respondentsprovided toeachofthethreepackagesarenotvisibleatthemoment theymakethechoice.Thereasonforthisisthatwewanted tostimulaterespondentstoonceagainconsiderthe techni-calmeasuressotheywouldnotonlybasetheirchoicesonthe ratingstheyjustprovided.However,respondentscould con-sulttheratingsbyscrollingbacktotheratingquestionsand theratingstheyprovided.

Tofurtherlimittheeffortexpectedfromrespondents,the constructed6choicesetswereblockedintotwoblocksofthree choicesetseach.Arespondentwasrandomlyassignedtoonly oneofthetwoblocks.Thus,intotaleachrespondentmade threechoices,andprovided18perceptionsratings:nine secu-rityandnineusabilityperceptionratings.

Aswasexplainedatthebeginning of Section3,theDCE inthisstudywasconstructedbecausetheemployees’choices amongsecuritypackagescannotbeobservedinreallife.As aconsequence,socalledstatedbehaviourisobserved,hence, whattherespondentssaytheywilldowhenthepresented hypotheticalchoicesituationbecomesareality.Although re-sponsesobservedinDCEsareoftencriticizedforthe possi-bilitythat statedresponsesdo notnecessarilyreflectwhat peopleactuallywilldoinreallife,validationstudies

(8)

gener-Fig.3– Exampleofthechoicetask.

allyshowhighlevelsofaccuracyinpredictingactualchoice behaviourbymeansofmodelsestimatedfromresponses ob-servedinDCEs(e.g.WlömertandEggers,2016).

4.4. Modelestimation

Randomutilitytheoryassumesthatdecisionmakers,inour caseemployees,choosethatalternativefrom asetof alter-natives fromwhichtheyderivethe highestutility.Itis fur-therassumedthattheyderiveacertainutilityfromeach at-tributelevel,inourcase,aTSMlevel.Thisutility-component iscalledapart-worthutility.Finally,itisassumedthatthese part-worthutilitiesarecombinedtoarriveatanoverall util-ityforanalternative.Althoughotherutilityspecificationsare possible,itistypicallyassumedthatthisprocesscanbe ap-proximatedbythefollowinglinearadditiveutilityfunction:

Uj=Vj+εj=

i

βiXi j+εj

where,U_jistheutilityderivedfromanalternativej,V_jisthe structuralorsystematicpartofutility,whichcanbepredicted bythemodel,ɛjistherandompartutility,whichisthepartof

utilitythatcannotbepredictedbythemodel(e.g.covering

id-iosyncrasiesfromthesideofthedecisionmaker),Xijdenote

theattributelevelsofattributeiforalternative j;andβi are

theweightsoftheattributesi,hence,theparametersthatare estimated.TheproductβiXijinvolvesthepart-worthutilityof

anattributelevel,i.e.,thecontributionmadebythatattribute leveltotheutilityofanalternative.Byassumingthattheerror termɛjisindependentlyandidenticallydistributedaccording

totheso-calledExtremeValueTypeIdistribution,choice prob-abilitiestakethefollowingMultinomialLogitform:

pj= eVj

keVk

wherepistheprobabilityofchoosingalternativejamonga setofalternativesk,andeisthebaseofthenaturallogarithm. ParameterestimatesareobtainedusingMaximumLikelihood Estimation routines. For a more detailed introduction into choicemodelingwerefertoBen-AkivaandLerman(1985).

Becauseall attributes inthis study are categorical,they needtobecodedfirstinordertobeincludedinmodels.For thisweappliedaso-calledeffectscodingscheme,whichis presentedinTable2(BechandGyrd-Hansen,2005).This cod-ingschemeinvolvesthattheLlevelsofanattributearecoded byL−1indicatorvariables.ThefirstL−1levelsarecoded1on

(9)

Table2– Effectscodedattributelevels.

Attributes Levels Parameters

PLMM PLM

Passwordlength(PL) Minimal8characters,1uppercaseletter,1 specialcharacterand1numericcharacter

1 0

Minimal8characters 0 1

Norestrictions −1 −1

PEFOQ PEFOY

Passwordexpiryfrequency(PEF) Onceaquarter 1 0

Onceayear 0 1

Never −1 −1

BR

Browserrestrictions(BR) Obligatorybrowser 1

Everybrowserisallowed −1

ERPM ERWM

E-mailrestriction(ER) Pop-upmessagewithe-mailwhich containsconfidentialwords

1 0

Warningmessagewithe-mail 0 1

Norestrictions −1 −1

FS

Filesharing(FS) Viacorporateshareddrive 1

Norestrictions −1

eachrespectiveindicatorvariableand0onallother indica-torvariables,whiletheLthlevel iscoded−1on all indica-torvariables.Ifallattributesareeffectscoded,thenan esti-matedconstantcanbeinterpretedasthemeanscoreonthe dependentvariablesderivedfromallevaluatedalternatives. EstimatedcoefficientsfortheL−1indicatorvariablesthen in-dicatetowhatextentthecorrespondinglevelsaffectthe de-pendentvariable.Bydefinition,thecontributionstothe de-pendentvariableofthelevelsthatbelongtothesameattribute summatetozero.Inutilitymodels,theparametersestimated fortheL−1indictorvariablesexpressthemarginalutilityof thecorrespondinglevel.ThemarginalutilityoftheLthlevel isthenegativesumofthemarginalutilitiesoftheotherL−1 levels

ThestructuralutilityVderivedfromalternativejcanbe specifiedasfollows:

V_j=βV

PLMM· PLMMj+βVPLM· PLMj+βVPEFOQ· PEFOQj

+βV

PEFOY· PEFOYj+βBRV · BRj+βERPMV · ERPMj

+βV

ERWM· ERWMj+βVFS· FSj+βPSV · PSj

+βV

PSQ· PS2j+βVPU· PUj+βPUQV · PU2j

whereβ are theparameters tobeestimatedandthe other termsareasexplainedin Table2,exceptforthePSandPU, whichdenotetheobservedperceivedsecurityandperceived usabilityratingsresepectively(note:thesewereobtainedper individualbasedontheoutcomesoftheratingtaskdescribed above).Becauseweexpectthatthemarginalincreasein util-ity diminisheswithhigherinitial perceptionlevels,weadd quadratic terms forPSand PU.Notethat because we con-ductedanunlabeledexperiment,noconstantisincludedin theutilityfunctionasthereisnoreasontoexpectthat respon-dentswouldsystematicallypreferthefirst,secondorthird al-ternativeinachoiceset.

Inadditiontothechoicemodel,weestimateseparate mod-elsforthesecurityandusabilityratingsthatareobservedfor everyalternative.Theseratingsareassumedtobeofinterval

measurementlevel,hence,regressionmodelsareestimated toexaminetowhatextenteachindicatorvariableaffectsthe perceivedsecurityandperceivedusability,respectively.More specifically,thefollowingfunctionisestimatedtopredictthe perceivedsecurityPSP_of_an_alternative_j:

PSP

j =C+βPLMMPS · PLMMj+βPLMPS · PLMj

+βPS

PEFOQ· PEFOQj+βPEFOYPS · PEFOYj

+βPS

BR· BRj+βERPMPS · ERPMj

+βPS

ERWM· ERWMj+βFSPS· FSj

Cistheregressionconstantandβ aretheparameterstobe estimated.Asimilarmodelisestimatedtopredictperceived usabilityPUP_(we_leave_out_the_{corresponding}_function_to_avoid

repetition).Becauseeffectscodingisappliedandthusall at-tributesareexpressedonthesamescale(−1to1),the esti-matedparametersintheperceivedsecurityandperceived us-abilityequationscanbedirectlycomparedintermsofweight, i.e.,theimpactontheobservedperception.Notethatthese parameterscannotbedirectlycomparedtotheparametersof thechoicemodels,becausetheseareexpressedonadifferent scale.

4.5. Datacollection

Thepopulationofinterestconsistsofallemployeeswhosejob involvesworkingwithcomputersonaregularbasis.Asample isrecruitedfromthispopulationbyapplyingsnowball sam-pling,startingwiththe personalnetworkofoneofthe au-thors.Eachrespondentwasaskedtosendthequestionnaire tothreepersonsoftheirsocialnetworkthatbelongedtothe population.Intotal,230respondentscompletelyfilledoutthe questionnaire.

Table3 presents adistribution ofrespondent character-istics.Thetablemakesclearthatmoremalesthan females responded. Furthermore, respondents are relatively young

(10)

Table 3 – Distribution or respondentcharacteristics (in percentagesofN=230). Gender Male 60.0 Female 40.0 Age <25years 20.9 25–29years 37.8 30–39years 19.1 40–49years 10.0 50+years 12.2 Companysize(numberofemployees) <10 6.5 10–49 10.0 50–249 10.9 250–500 6.1 500–999 5.2 1000–9999 31.3 10,000+ 30.0 Workexperience <1year 41.3 1–4years 34.4 5–9years 11.8 10+years 12.6 Shareworktimeoncomputer 0–25% 3.0 26–50% 7.0 51–75% 23.0 76–100% 67.0

(averageageis32years),whichisalsoreflectedinthe rela-tivelylargeshareofrespondentwithrelativelylimited num-berofyearsofworkexperience.Ontheotherhand,twothirds ofthe respondentsspendmostoftheir workingtimeon a computer,sointhatrespecttheyareexperienced.Finally,the resultsshowthatthefarmajorityoftherespondentsworksin bigofverybigcompanies.

Becauseofthenon-randomstartingpointofthesnowball procedure,thequestionnairecannotbeconsideredarandom sample.Socareshouldbetakentogeneralizetheresultsfrom thissampletothewiderpopulationofemployees.

5. Results

Inthissection,wepresentanddiscusstheresultsofthethree estimatedmodels.Thissectionisorganizedbyfollowingthe fourearlierformulatedresearchquestions.

5.1. Securityandusabilitycorrelation

Westartbyfocussingonthefirstresearchquestion:Do per-ceived usability andperceived securitycorrelatenegatively, assuggestedintheLiterature?Thisexpectationcanindeed beconfirmedbytheempiricalresults:thecorrelationis nega-tive,−0.143(p=0.000),albeitsuggestingarelativelyweak as-sociation.Thisfindingontheonehandconfirmsnotionsfrom theliteraturethatonanaveragehigher(perceived)securityis pairedwithlower(perceived)usability.Ontheotherhand,the relativelylowcorrelationalsoindicatesthatthisisnot neces-sarilyalwaysthecase,assuggestedintheliteratureaswell. Hence,thissuggeststhatitshouldinprinciplebepossibleto designtechnicalsecuritymeasuresthatarebothperceivedto besecureandusable(notethatourresultsprovidesome op-tionstodoso,whichwewilldiscusslater).

Table4– Distributionsofobservedusabilityandsecurity ratingsandcorrelation(N=2070).

Rating Security Usability

1 9.1% 2.1% 2 32.0% 13.3% 3 24.8% 29.3% 4 28.0% 44.0% 5 6.0% 11.2% mean 2.90 3.49 median 3.00 4.00 stand.dev. 1.094 0.932 correlation −0.143

Table4presentsthedistributionsfortheobservedsecurity andusabilityperceptionratings.Theresultsindicatethatfor bothsecurityandusabilitythefullrangeoftheratingscale isusedbyrespondents.Comparingthedistributionsreveals thatonaveragethepresentedsecuritypackagesscorehigher onperceivedusabilitythanonperceivedsecurityandthatthe spreadinusabilityratingsissomewhatsmaller.

5.2. Securityandusabilityperceptionoftechnicalsecurity measures

Toanswerthesecondresearchquestion(Aremorerestrictive measuresperceivedasmoresecureandlessuserfriendly?), weinspecttheresultsofthetworegressionmodelsestimated fromtheobservedperceptions.ThesearepresentedinTable5:

inthefirstcolumntheparametersofPerceivedSecurity(βPS_),

inthesecondcolumnthoseofPerceivedUsability(βPU_).

Re-callfrom Table3thatwe estimateL−1 parametersfor theL attributelevelsofeachattribute.Absolutet-values>1.96 de-noteastatisticallysignificantparameterattheconventional 95%confidencelevel.Inordertogiveafullpictureandtoease interpretationofthelevelsofallvariedsecurityattributes,we addedtheeffectoftheLthleveltothetable(initalics).The lattereffectsare notestimatedbutderived:becauseeffects codingisapplied,thecontributionstotheratingsofalllevels ofanattributesumtozerobydesign.Theseeffectsarethus expressedindeviationsfromtheaverage,whichinboth re-gressionmodelsisdenotedbytheestimatedconstant.Model fitofthePerceptionmodelsisbasedonthe well-known R-squaremeasurewhichgives thepercentageofvariationin perceptionwhichisexplainedbythemodel.Modelfitofthe ChoicemodelsismeasuredbasedonMcFadden’srho-squared (e.g. Ben-Akiva&Lerman,1985),whichgivesthepercentage ofinitial uncertainty – from theside ofthe analyst– con-cerning choiceprobabilitieswhich iseliminatedby the es-timatedmodel. Bothrange from 0to1,withhighervalues indicatingabettermodel-fit.TheRho-squaredvaluesofthe presentedmodel(see Table5)are0.25and0.44respectively, whichmanyresearchersinterpretasareasonablemodelfit and reasonablygood modelfit respectively(Hensheret al., 2005).Furthermore,both estimated choicemodels are sta-tisticallysignificant inthesense thattheyfit thedata bet-terthanthenullmodel:LL_Null-model=−758.04;LL_Model C (attributes only)=−565.66 (LRS=384.76, df=8, p=0.000);

(11)

Table5– Estimatedparametersandassociatedt-values(t>1.96impliessignificanceatthe5%level).

Perception Choice

A B C D

Security Usability Attributesonly Attributes+perceptions

βPS _t _βPU _t _βV _t _βV _t Regressionconstant(C) 2.90 156.51 3.49 185.52 Passwordlength Min8ch.,1uppercase,1 specialch.,1numericch. (PLMM) 0.58 20.06 −0.05 −1.75 0.89 11.28 −0.11 −1.14 Minimal8characters(PLM) 0.02 0.73 0.06 1.91 −0.02 0.23 0.57 5.89 Norestrictions −0.60 −0.01 −0.87 −0.46

Passwordexpiryfrequency

Onceaquarter(PEFOQ) 0.42 15.92 −0.24 −8.89 0.31 4.78 −0.03 −0.30 Onceayear(PEFOY) 0.02 0.83 0.12 4.43 0.11 1.46 0.28 3.46

Never −0.44 0.12 −0.42 −0.25

Browserrestrictions

Obligatorybrowser(BR) 0.04 1.83 −0.27 −13.28 −0.35 7.23 −0.22 −3.98

Everybrowserisallowed −0.04 0.27 0.35 0.22

E-mailrestriction Pop-upmessagewith

e-mailwhichcontains confidentialword(ERPM)

0.21 7.42 −0.14 −4.88 0.02 0.21 0.03 0.43

Warningmess.withe-mail (ERWM)

0.14 5.15 −0.06 −2.39 0.09 1.35 −0.07 −0.73

Norestrictions −0.35 0.20 −0.11 0.04

Filesharing

Viacorporateshareddrive (FS)

0.27 13.40 −0.08 −3.76 0.19 3.86 0.05 0.78

Norestrictions −0.27 0.08 −0.19 −0.05

Perceivedsecurity(PS) 2.51 5.74

Perceivedsecurity2_(PSQ) ₋₀_.24 ₋₃_.62

Perceivedusability(PU) 2.33 4.37

Perceivedusability2_(PUQ) ₋₀_.19 ₋₂_.57

Modelfit: R2₌_0.41 _R2₌_0.16 _Rho2₌_0.25 _Rho2₌_0.44

LL_ModelD(attributes+perceptions)=−425.37(LRS=665.34, df=12,p=0.000).

Basedonourdiscussionofpreviousresearch,weexpect thatincreasedrestrictionsareperceivedasmoresecurebut aslessusable(lessuser-friendly),hence,theireffectsare ex-pectedtohaveoppositesigns.Indeed,theresultssuggestthat thisisthecase:

• Havingmorerestrictionsonpasswordsisclearlyperceived toincreasesecurity(seecol.A)andthiseffectisrelatively large.Ontheotherhand,itseffectonusability(seecol.B) isnotstatisticallysignificant.

• Obligatorychangeofpasswordevery3monthsisperceived toimprovesecurity(seecol.A),whereasitisperceivedas lessusable(seeCol.B).

• Obligatorybrowserisperceivedtoimprovesecurity(col.A), althoughitseffectsisrathersmall,andisperceivedtobe lessusable(col.B).

• Alsoobligatory filesharing via a corporatedriveis per-ceivedtoimprovesecurity(col.A),butisperceivedasless usable(col.B).Theimpactonsecurityismuchlargerthan onusability.

• Finally, withrespecttoE-mailrestrictions,bothwarning messagesareperceivedtoincreasesecurity(col.A)andto decreaseusability(col.B).

Somefurtherresultsoftheregressionmodelsare notewor-thymentioning.ComparingtheR2_’s_of_the_two_models

indi-catesthattheproportionexplainedvarianceofthePerceived SecurityModel(col.A)ismuchhigherthanofthePerceived UsabilityModel(col.B).Hence,securityperceptioncanbe pre-dictedwithmoreprecisionthanusabilityperception. Possi-bly,interactionsbetweenattributesplayabiggerroleinthe usabilitymodeland/oremployeesaremuchmore heteroge-neousintheirusabilityperceptionsofsecuritymeasuresthan intheirsecurityperceptions.Furthermore,asreportedearlier andalsodenotedbythehigherregressionconstant,the aver-ageperceivedusabilitylevelofthepresentedtechnical secu-ritypackagesishigherthantheiraverageperceivedsecurity level.

5.3. Impactofsecurityandusabilityonchoice

Wenowfocusonthethirdresearchquestion:doesperceived usabilityorperceivedsecurityweighstrongerinemployees’

(12)

Fig.4– Utilitycontributionforsecurityandusabilityperception.

preferencesforcomputersecurity?Toanswerthisquestion, we inspect the parameters forperceived security and per-ceivedusabilityasestimatedbythemultinomiallogitmodel, which arepresentedincol.D of Table5.Asexpected,both (linear)parametershaveapositivesign,whichindicatesthat the morethesecurity measuresareperceived tobesecure andthemoretheyareperceivedtobeusable,themoreutility isderivedfromthepackagecontainingthesemeasures,and thusthemorelikelythatpackageischosen(ceterisparibus). Inadditiontotheparametersforthelineareffects,alsothe parametersforthequadraticcomponentsoftheperception ratingsarestatisticallysignificant.Theseparametersare neg-ative,whichsuggeststhatforhigherinitialvaluesofperceived securityandperceivedusability,furthermarginalincreasein utilityisdiminished.Thisisaplausibleoutcome:thehigher theevaluationofapackageoftechnicalsecuritymeasures al-readyis,thelessadditionalutilityisderivedfromafurther increase.Thiseffectisillustratedin Fig.4,whichpresentsthe utility contributionforpredicted security andusability rat-ings,asafunctionoftheinitiallevels.

Fig.4alsodemonstratesthattheimpactofperceived secu-rityandperceivedperceptiononutilityisaboutthesame.The figuresuggeststhatatlowervalues,theimpactofperceived securityislittlestronger,whileathighervaluestheimpactof perceivedusabilityisalittlestronger.Thissuggeststhatonce securityisatahighlevel,thusthepackageisconsideredsafe, usabilitybecomesmoreimportant.However,thedifferences foundinthesampleareverysmallandtheestimated param-etersdonotdifferinastatisticallysignificantway.Thus,we concludethatperceivedsecurityandperceivedusabilityaffect choiceofsecuritypackagestoasimilarextent.

ComparingRho-squarevaluesoftheMNLwithand with-outtheperceptionratingsasexplanatoryvariables(col.Cwith col.Din Table5)indicatestowhatextenttheratings them-selvesaffectchoice,beyondtheeffectsofthefactorsthat in-fluencetheseratings(i.e.,theTSMs).Byaddingthe percep-tionratings,theRho-squarevaluesignificantlyincreasesfrom 0.25to0.40,indicatingasubstantialimprovementofmodelfit. Hence,theobservedperceptionsplayasubstantialroleinthe choiceofthepreferredsecuritypackage.Asexpected, param-etersandt-ratiosincol.Daremostlysmallerthanthosein col.C,indicatingthatpartoftheireffectismediatedbythe securityandusabilityperceptions.

5.4. Directversusindirecteffectsofsecuritymeasures

Thislastresultraisedthefourthresearchquestion:Areall ef-fectsoftechnicalsecuritymeasuresonchoicemediatedby perceivedusabilityandperceivedsecurity?Iftheeffectsofthe securityattributeswouldallbemediatedbythetwo percep-tions,theirparameterswouldnotbestatisticallysignificance oncetheobservedperceptionswereincludedinthemodel. Hence,non-significantparameterssuggestthatthedirect ef-fectsofTSMsonchoicesarenon-existentandalleffectsare mediatedinanindirect process,i.e.,through theeffectsof TSMsonperceptionsandtheeffectsofperceptionsonchoice. Astheresultspresentedinthecol.DofTable5indicate,this is notthecase:evenwhencontrolledforsecurityandusability perception,thefollowingparametersofthetechnicalsecurity measuresonutilityarefoundtobestatisticallysignificant:

• Thelevelminimal8characterspasswordlength,ispreferred abovemorerestrictedlevels,thusthelessrestricted pass-wordrequirementispreferred.

• Thelevelonceayearofpasswordchangefrequencyismore preferredthanonceamonth,thusthelessfrequentchange ispreferred.

• Every browser allowed is more preferred than obligatory

browser,thusthelessrestrictedlevelispreferred.

Whatmaybeconsideredremarkable,isthatthese signif-icant levelsall concernless restrictivemeasures.Moreover, theyallconcern levelsofwhichwefoundearlierthatthey positivelyinfluenceperceivedusabilityratings.Ontheother hand,mostofthelevelsofwhichweearlierfoundthatthey increasedthe perceivedsecurityratingslose theirstatistical significance.Incontrast,theselevelsallsignificantlyincrease utilityintheattributes-onlyMNLmodel(seecol.CofTable5), i.e.themodelwhichdoesnotincludetheperceptions.These resultssuggestthatperceivedsecuritymediatessecurity re-latedaspectsofthetechnicalsecuritymeasures,whereas per-ceived usability doesnot fully mediateusability aspectsof theseTSMs.

Thequestionisthenwhatthethreesignificantdirect ef-fectsrepresent;inotherwords,couldtheseforexample rep-resentanother(perception)dimensioninadditiontosecurity andusability?Wecanonlyspeculateaboutthis,becausewe

(13)

donothaveadditionalmeasurements.Apossibilityisthatthe threesignificantlevelsrepresentcurrentsecuritylevelsmany employeescurrentlyexperienceatwork.Suchathird dimen-sionthatmightplayaroleinpreferencesofsecuritypackages inadditiontosecurityandusabilitycouldbelabelledas famil-iaritywiththesecuritymeasuresatwork.Anotherpossible di-mensionmayberelatedtoaTSM’simpactonthebusinessat largeratherthan(individual)usability.Forexample, employ-eesmayprefermeasuresthatareknowntosecurehighly im-portantbusinessresources,ortheymayprefermeasureswith limitedimpactonoverallproductivity.

5.5. Anillustration

In this section,anillustration ofthe resultsisprovided to demonstratehowtheestimatedmodelscanbeappliedto pre-dictemployees’perceptionsandpreferencesconcerning dif-ferentsecuritypackages(i.e.,combinationsofTSMs).This ap-plicationshowshowthemodelcanbeusedbyCISOsinthe designofsecuritypackages,forexample,todesignan opti-malsecuritypackage.Wefirstapplythemodeltopredictthe choiceprobabilities forascenarioinwhich employeescan onlychoosebetweenausabilityoptimalandasecurity opti-malpackage:(1)the“usabilityoptimal” packagemaximizes the user-friendliness andconsistsofthose TSM levelsthat allcontributehighesttoperceivedusability,whichallinvolve lessrestrictivemeasures(see Table5);(2)the“security opti-mal” package maximizessecurity andconsistsofthemost restrictivelevelsofeachTSMthatallcontributedhighestto perceivedsecurity.Table6presentsthelevelsofthepackages andtheircontributionstoperceivedsecurity,perceived usabil-ityandutilitycontributionbasedontheparameterestimates presentedinTable5.

Topredictchoiceprobabilities,wefirstneedtopredictthe utilitiesofbothpackagesthatconsistofdirectandindirect effectsofthetechnicalsecuritymeasures.Toillustratethis, wecalculatetheutilityofthefirstpackage.Completelyinline withearlierpresentedequations,thecontributiontoutilityof thedirecteffectsissimplythesumofdirecteffects,whichare presentedinthelastcolumnofTable6(thedirecteffectofthe firstpackageis1.06).Tocalculatetheindirecteffects,wefirst needtopredictthesecurityandtheusabilityperceptionsof thepackage,whichcanbefoundbysummingtheresultsin thefirstandsecondcolumnsofTable6,respectively.The util-itycontributionofperceivedsecurityandperceivedusability totheoverallutilityofthesepackagesisthencalculatedby weighingthepredictedperceptionvalueswiththeir parame-tersasestimatedbytheMNLmodel(theattributesplus per-ceptionmodel).TheutilitycontributionofPSinthefirst pack-age=2.51∗2.28−0.24∗_2.282₌_4.48;_the_utility_contribution_of

PU=2.33∗4.22−0.19∗_4.222₌_6.25._These_utility_{contributions}

representtheindirecteffectoftheTSMsmediatedbythe per-ceptions.Theoverallutilityofthepackageisasummationof thetwoindirecteffectsandthedirecteffect(11.98).

Inasimilarfashion,theoverallutilityofthesecurityoptimal

packagecanbecalculated(11.04).Forthescenarioinwhich employeescanonlychoosebetweentheusabilityoptimaland thesecurityoptimalpackages,theMNLmodelpredictsthat (exp(11.98)/(exp(11.98)+exp(11.04))=)72% ofthe employees would choosetheusability optimalpackageand,hence,28%

wouldchoosethe securityoptimalpackage.Hence,the large majoritywouldnotpreferthesecurityoptimalpackage.

Assumethat the CISO wishestodesignahighly secure packagethatismorepreferredbytheemployees,forexample, becauseshebelievesthiswouldincreasecomplianceandless counter-effectivebehaviour of employees.Hence, the CISO wishestokeepahighsecuritypackagesandthereforeonly al-lowsaminimalconcessiontouser-friendliness.Sheassumes thefollowingpackage:(3)“jointoptimal”,whichhasthesame highsecuritylevelsasthesecurityoptimalpackage,except thatforbrowserrestrictionsthelevelobligatorybrowseris re-placedbythemoreuser-friendlyleveleverybrowserallowed. Theresultsindicate thatthis adaptationhardlyaffects the

securityperception,butitconsiderablyincreasestheusability perceptionandresultsinahigherdirectutilitycontribution. Thisresultsin anevenhigher overall utility(12.10) ofthis packagethantheusabilityoptimalpackage.Ifemployeescould onlychoosebetweenthe usability optimaland thejoint opti-mal package,theMNL model predictsthat 53% would

pre-fer thejoint optimalpackage,and 47%would prefer the

us-abilityoptimalpackage.Hence,insteadofonly28%preferring thehighlysecurepackage,now53%oftheemployees,thus themajority,prefersthehighlysecurepackageoverthemost user-friendlypackage,whileonlyasingleconcessionismade touser-friendliness.

ThisexamplesuggeststhatCISOscandesignand imple-mentahighlysecuritypackagethatisstillpreferredbya ma-jorityoftheemployees.Thispackageinvolvesamaximumof passwordrestrictions,frequentpasswordchanges,filesharing viaashareddriveandemailrestrictionsthatinvolve warn-ingmessages.Obligatorybrowsers,ontheotherhand,arenot supportedbytheemployees,sotheyarenotincludedinthe package:this TSMisnotperceivedtocontributetosecurity whileitisregardedaslessuser-friendly.Itgoeswithout say-ingthatthemoreconcessionsaremadetouser-friendliness, themoreemployeeswillprefer theresultingsecurity pack-ages.Asisdemonstratedhere,CISOscanapplythemodelto exanteevaluatedifferentsecuritypackagedesignsinterms ofemployeespreferencesandinthiswaydesigntheiroptimal securitypackage.

6. Conclusion

and

discussion

Inthis paper,employees’preferencesfortechnicalsecurity measuresthatcompaniescantaketoprotectinformationare studiedwithin theempiricalframeworksofdiscretechoice theoryanddiscretechoiceexperiments.Morespecifically,an experimentisconducted,inwhichemployeesevaluate com-binationsoftechnicalsecuritymeasuresintermsofsecurity andusabilityperceptionsandmakechoicesamongsecurity packages.Regression models were estimated from the ob-servedperceptionratings,theparametersofwhichexpress towhat extentsecurity measuresaffect perceived security andperceived usability.Inaddition,aso-calledMNLmodel (beingtheworkhorsemodelfordiscretechoiceanalysis)was estimatedfromtheobservedchoices,whichrevealedthe rel-ativeimpactofsecurityandusabilityperceptionsonchoice. Ourresultsprovideinsightintothetrade-offmadebyusersof

(14)

Table6– Anillustration:predictedemployeeresponsestothreesecuritypackages.

Contributions

PS PU V

Package1 “usabilityoptimal”

Regressionconstant 2.90 3.49

Passwordlength Minimal8characters 0.02 0.06 0.57

Passwordexpiry Onceayear 0.02 0.12 0.28

Browserrestrictions Everybrowserisallowed −0.04 0.27 0.22

E-mailrestriction Norestrictions −0.35 0.20 0.04

Filesharing Norestrictions −0.27 0.08 −0.05

Predictedperceptions 2.28 4.22

Predictedutilitycontribution 4.48 6.45 1.06

Overallutility 11.98

Package2 "securityoptimal"

Passwordlength Min8ch.,1upperc.1sp.ch., 0.58 −0.05 −0.11

Passwordexpiry Onceaquarter 0.42 −0.24 −0.03

Browserrestrictions Obligatorybrowser 0.04 −0.27 −0.22

E-mailrestriction Pop-up– confidentialwords 0.21 −0.14 0.03

Filesharing Viacorporateshareddrive 0.27 −0.08 0.05

Predictedutilitycontribution 6.41 4.92 −0.28

Package3 "jointoptimal"

Passwordlength Min8ch.,1upperc.1sp.ch., 0.58 −0.05 −0.11

Passwordexpiry Onceaquarter 0.42 −0.24 −0.03

Browserrestrictions Everybrowserisallowed −0.04 0.27 0.22

E-mailrestriction Pop-up– confidentialwords 0.21 −0.14 0.03

Filesharing Viacorporateshareddrive 0.27 −0.08 0.05

Predictedutilitycontribution 6.37 5.57 0.16

Choiceprobability A B

A=package1 B=package2 72% 28%

A=package1 B=package3 47% 53%

information technology, between security and user-friendlinessaspectsoftechnicalsecuritymeasures.

Basedontheresultsoftheestimatedmodels,answersare formulatedtofourresearchquestions,whichcanbe summa-rized asfollows.First,perceived usabilityandperceived se-curityindeedcorrelatenegativelyasissuggestedinthe lit-erature, although we find that the association is relatively weak(−0.14).Second,asexpected,morerestrictivesecurity measuresare perceived asmoresecureand asless usable. Third, perceived security and usability affect choice tothe same extent;that is,bothdimensionsoftechnicalsecurity measures areconsidered equally importantbyusers of in-formation technology.Asexpected,higher securityand us-abilityperceptionscoresincreasethepreferenceforsecurity packages;however,andinlinewithintuition,themarginal in-creasediminisheswithhigherinitiallevelsofsecurityand us-abilityperceptions.Fourth,perceivedsecurityfullymediates theeffectofsecurityrelatedaspectsoftechnicalsecurity mea-sures,whileperceivedusabilitydoesnotfullymediatethe ef-fectsofuser-friendlinessrelatedaspectsofsecuritymeasures. Theresultsgiverisetothepossibilitythatotherdimensions

existsthatmediatetheeffectsofTSMs,suchasforexample familiarity.However,thispossibilityneedsfurtherresearch.

Ourfindingsthat(a)employeesclearlyrecognizethatmore restrictivemeasuresimprovesecurity,and(b)securityis con-sideredbythemtobeequallyimportantasusability,may en-courageCISOsofcompaniestoadoptamorecooperative pro-cessintheirsecuritydesignprocess,inwhichperceptionsand preferencesofemployeesaretakenintoaccount. Investigat-ingemployeepreferences,likeinourstudy,mayleadtothe de-signandimplementationofpackagesofsecuritycontrolsthat are better tailored towardsemployee’s needs,reducing cir-cumventionactivitiesthatcouldbeexploitedincyberattacks. Weprovidedanillustrationofhowthemodelsestimatedin thisstudycanbeappliedforthispurpose.However,thiswill notbesimplyamatterofselectingtherightcontrols;itwill alsoinvolveproperlymanagingcommitmentandawareness. Interestingavenuesforfurtherresearchwithinthediscrete choiceframeworkincludethefollowing.First,thenumberof technicalsecuritymeasuresincludedinourstudywasrather limited(forgood reasons).Hence,itwould beofinterestto includemoreofthosemeasures,suchasforexample, multi-factorauthentication,andexaminewhetherthestrengthof

(15)

the correlationbetweenperceived securityand usabilityas foundinthisstudyisrobust.Second,inourstudyperceptions aremeasuredfirst,andthenchoicesareobserved.The ques-tioniswhetherexplicitlyaskingaboutusabilityandsecurity firstmakesrespondentsmoreconsciousoftheseaspects(i.e., increasestheir salience),sothe issueistowhatextentthe presentationorderaffectedtheresults.Itwouldbeof inter-esttostudytowhatextentourresultsarerobustundera dif-ferentorderofbothmeasurementtasks.Third,thepossibility ofotherdimensionsinadditiontosecurityandusability,e.g. familiarity,couldbefurtherinvestigated.Fourth,theresults presentedinthispaperwerebasedonaconvenience sam-ple,andshouldthereforebetreatedwithcare.Hence,further researchshouldincludemorerepresentativesamples.Fifth, heterogeneityinperceptionsandpreferencecouldbe exam-ined.Thediscretechoiceparadigmoffersarange of meth-odstostudyheterogeneity(Greene&Hensher,2003),ofwhich thefollowingthreeareprobablymostpromisinginthe con-textofresponsetoinformationsecuritymeasures.First, tra-ditionalsegmentationcouldbeapplied,whichimplies exam-iningtowhatextentpeoplewithdifferentsociodemographic characteristicsdifferintheirperceptionofandpreferencesfor TSMs.Second,itcanbeassumedthatpreferenceweightsdo nothavecrispvaluesbutfollowacertaindistributionacross employees,whichcanbeexaminedbyestimatingmore ad-vancedchoicemodels,suchasmixedlogitmodels.Third, la-tentclassesmaybeassumed,whicharegroupsinthe pop-ulationthatareinternallyhomogeneousintheirpreferences andwhichcanbeidentifiedbasedontheirobservedchoices. Inthesemodels,membershipfunctionscanbeestimatedthat allowpredictingtheprobabilityofbelongingtoalatentclass basedonobservedindividualcharacteristics.

Apartfromextensionstothepresentstudy,itishopedthat thispaperstimulatesotherchoicemodellingapplicationsin thisfield,bothextendingtheworkonemployeepreferences aswellasfocusingonthechoicesofotheractorsinthe cyber-securityplayingfield.Intermsofemployees,thismaynotonly involve studying preferencesforsecurity controls,but also choicesintermsofcomplianceornon-compliancewith secu-ritypolicies.Choicesfornon-compliancemayhappen sponta-neously,forexamplewhenofficialsecurityisfoundtoo cum-bersome,orinresponsetodeceptiveactsofattackers,such asinphishing(FinnandJakobsson2007)orsocialengineering (Bulléeetal.2015)attacks.Howattributesofpoliciesand situa-tionscontributetopreferencesfor(non-)compliancemayhelp inimprovingorganizationalaspectsofsecurity.Onepossible applicationtootheractorsliesinanalysingthechoices secu-rityofficersmakewhenselectingcontrolstobeimplemented intheirorganization.Whichattributescontributetothe util-ityofapossiblecontrol,andhowdoesthisaffectthedecision? Anotherpossibilityistostudychoicesofcyber-attackers,in termsofwhichtargetstoattackusingwhichmeans, assum-ingthattherearesubjectswillingtoparticipate,eitherknown offendersorwhite-hat(ethical)hackers.Betterunderstanding ofattackerchoicesmayinformbetterrepresentationsof at-tackerbehaviourinsecuritymodelsandriskanalyses.Inthese ways,discretechoicetheoryanddiscretechoiceexperiments maybecomeusefultoolsintheportfoliooftechniquesfor im-provingsecurityincyberspacebyconsideringthehuman fac-tor.

Asafinalnote,thereisadebatearoundhowmuch con-trolshouldactuallybegiventoemployeesregardingsecurity choices.Muchofthe existing practices assumecentralized controlofsecuritysolutions(cf.Parkin,Kassab&VanMoorsel 2008),butonecouldimagineframeworksinwhich employ-eescandecide howmuchsecuritythe dataorapplications theyworkwithrequire.Thisso-called“laissez-fairesecurity” (Johnsonetal.2009)requiresinvestigationnotjustofthe pref-erencesofemployeeswithrespecttotechnicalsecurity mea-sures,butalsoregardingtheirpreferredlevelofcontroloversuch measures.

R E F E R E N C E S

AdamsA, SasseMA.Usersarenottheenemy.CommunACM 1999;42(12):40–6.

Andersson,D.(2013).Authenticationwithpasswordsand passphrases:implicationonusabilityandsecurity.

http://www.rlvision.com/blog/authentication- with-passwords-passphrases-implications-on-usability-and-security/

BeautementA, ColesR, GriffinJ, IoannidisC, MonahanB, PymD, etal.Modellingthehumanandtechnologicalcostsand benefitsofUSBmemorysticksecurity.Managinginformation riskandtheeconomicsofsecurity.US.:Springer;2009. p.141–63.

BechM, Gyrd-HansenD.Effectscodingindiscretechoice experiments.HealthEcon2005;14(10):1079–83.

Ben-AkivaM, LermanSR.Discretechoiceanalysis:theoryand applicationtotraveldemand.Cambridge:MITPress;1985. BrostoffS, InglesantP, SasseMA.Evaluatingtheusabilityand securityofagraphicalone-timePINsystem.Proceedingsof the24thBCSinteractionspecialistgroupconference.British ComputerSociety;2010.p.88–97.

BulléeJWH, MontoyaL, PietersW, JungerM, HartelPH.The persuasionandsecurityawarenessexperiment:reducingthe successofsocialengineeringattacks.JExpCriminol 2015;11(1):97–115.

CaputoDD, PfleegerSL, SasseMA, AmmannP, OffuttJ, DengL. Barrierstousablesecurity?Threeorganizationalcasestudies. IEEESecurPriv2016;14(5):22–32.

CatuognoL, GaldiC.Analysisofatwo-factorgraphicalpassword scheme.IntJInfSecur2014;13(5):421–37.

CranorLF, GarfinkelS.GuestEditors’Introduction:secureor usable?IEEESecurPriv2004;2(5):16–18.

DavisJrFD.Atechnologyacceptancemodelforempirically testingnewend-userinformationsystems.Massachusetts InstituteofTechnology,1986.

DhillonG, OliveiraT, SusarapuS, CaldeiraM.Decidingbetween informationsecurityandusability:developingvaluebased objectives.ComputHumBehav2016;61:656–66.

Dinev,T.,J.GooandK.Nam(2006),Userbehaviourtoward preventivetechnologies– culturaldifferencesbetweenthe UnitedStatesandSouthKorea.In:Proceedingsofthepaper presentedattheECIS.

FinnP, JakobssonM.Designingethicalphishingexperiments. IEEETechnolSocMag2007;26(1):46–58.

FurnellS.Theusabilityofsecurity-revisited.ComputFraudSecur 2016;2016(9):5–11.

Gandal,S.(2015).Lloyd’sCEO:cyber-attackscostcompanies$400 billioneveryyear,

http://fortune.com/2015/01/23/cyber-attack-insurance-lloyds, (AccessedMarch,15,2017).

(16)

GreeneWH, HensherDA.Alatentclassmodelfordiscretechoice analysis:contrastswithmixedlogit.TranspResPartB Methodol2003;37(8):681–98.

GutmannP, GriggI.Securityusability.IEEESecurPriv 2005;3(4):56–8.

HagenJM, AlbrechtsenE, HovdenJ.Implementationand effectivenessoforganizationalinformationsecurity measures.InfManagComputSecur2008;16(4):377–97. HensherDA, RoseJM, GreeneWH.Appliedchoiceanalysis:a

primer.CambridgeUniversityPress;2005.

HerleyC.Solong,andnothanksfortheexternalities:therational rejectionofsecurityadvicebyusers.Proceedingsofthe workshoponnewsecurityparadigmsworkshop,2009.

ISACAandRSAConference(2015),StateofCybersecurity: Implicationsfor2015.AnISACAandRSAConferenceSurvey, ISACAandRSAConference.https://www.isaca.org/cyber/ Documents/State-of-Cybersecurity_Res_Eng_0415.pdf

(AccessedMarch,15,2017).

JohnsonML, BellovinSM, ReederRW, SchechterSE.Laissez-faire filesharing:accesscontroldesignedforindividualsatthe endpoints.Proceedingsofthenewsecurityparadigms workshop.ACM;2009.p.1–10.

KaindaR, FlechaisI, RoscoeAW.Securityandusability:analysis andevaluation.ProceedingsoftheARES’10international conferenceonavailability,reliability,andsecurity.IEEE;2010. p.275–82.

KirlapposI, ParkinS, SasseMA.Shadowsecurityasatoolforthe learningorganization.ACMSIGCASComputSoc

2015;45(1):29–37.

LouviereJJ, HensherDA, SwaitJD.Statedchoicemethods: analysisandapplication.Cambridge:CambridgeUniversity Press;2000.

ManskiCF.Thestructureofrandomutilitymodels.TheoryDecis 1977;8(3):229–54.

McFaddenD.Economicchoices.AmEconRev2001;91(3):351–78. MohamedMA, ChakrabortyJ, DehlingerJ.Tradingoffusability

andsecurityinuserinterfacedesignthroughmentalmodels. BehavInfTechnol2016:1–24.

NurseJR, CreeseS, GoldsmithM, LambertsK.Guidelinesfor usablecybersecurity:Pastandpresent.Proceedingsofthe thirdinternationalworkshoponcyberspacesafetyand security(CSS).IEEE;2011.p.21–6.

ParkinSE, KassabRY, VanMoorselA.Theimpactofunavailability ontheeffectivenessofenterpriseinformationsecurity technologies.Proceedingsofthefifthinternationalservice availabilitysymposiumonserviceavailability,ISAS2008. Springer,2008.

PostGV, KaganA.Evaluationinformationsecuritytradeoff: restrictingaccesscaninterferewithusertasks.ComputSecur 2007;26(3):229–37.

RoseJ, BliemerM.Constructingefficientstatedchoice experimentaldesigns.TranspRev2009;29(5):587–617. SandersWH.Quantitativesecuritymetrics:unattainableholy

grailoravitalbreakthroughwithinourreach?IEEESecurPriv 2014;12(2):67–9.

SchultzEE.Researchonusabilityininformationsecurity.Comput FraudSecur2007;2007(6):8–10.

ShengS, BroderickL, KorandaCA, HylandJJ.WhyJohnnystill can’tencrypt:evaluatingtheusabilityofemailencryption software.Proceedingsofthesymposiumonusableprivacy andsecurity;2006.p.3–4.

VenkateshV, DavisFD.Atheoreticalextensionofthetechnology acceptancemodel:fourlongitudinalfieldstudies.ManagSci 2000;46(2):186–204.

WlömertN, EggersF.Predictingnewserviceadoptionwith conjointanalysis:externalvalidityofBDM-based

incentive-alignedanddual-responsechoicedesigns.Market Lett2016;27:195–210.

EricMolinisanassociateprofessorofTravelBehaviorResearch attheEngineeringSystemsandServicesDepartmentofthe fac-ultyTechnology,PolicyandManagement,TUDelft.Heconducts researchthatisonthecrossingbetweenapplyingcuttingedge behavioralresearchmethodsandgeneratingpolicyrelevant in-sightsforemergentpolicytopics.Heisanexpertindeveloping (advanced)statedchoiceexperiments.Topicsofresearchinvolve amongothers technologyandpolicyacceptancemainlyin the fieldofTransportation.Heco-chairsthesubcommitteeonstated responsetravelsurveymethodsoftheTransportationResearch Board,WashingtonDC.

KirstenMeeuwisseisaconsultantinCyberSecurityworkingat Deloitte.ShegraduatedfromtheTUDelftoftheMasterprogram inSystems,Engineering,PolicyAnalysisandManagement.Her thesisresearchwasaboutthetrade-offbetweensecurityand us-ability.Heraimistomakesecuritycontrolsuser-friendly.Inthat wayend-usersarenotannoyedbyworkingwiththesecontrols andthereforewillnotcircumventthesesecuritymeasures,which leadstoamorecybersecureworld.

WolterPietersisanassociateprofessorincyberriskatDelft Uni-versityofTechnology,facultyofTechnology,Policyand Manage-ment.HehasMScdegreesincomputerscienceandphilosophy ofscience,technologyandsocietyfromtheUniversityofTwente, andaPh.D.ininformationsecurityfromRadboudUniversity Ni-jmegen,focusedonthecontroversyonelectronicvotingin elec-tions.Hisresearchinterestsincludecyberriskmanagement, cy-bersecuritydecisionmaking,andcyberethics.Hewastechnical leaderoftheTREsPASSEuropeanprojectonsocio-technicalcyber riskmanagement,andiscurrentlypartoftheCYBECOprojecton behaviouralmodelsforcyberinsurance.

CasparChorusisProfessorofChoicebehaviormodelingatthe FacultyofTechnology,PolicyandManagement.Hismainresearch aimistoincreasethebehavioralrealismofchoicebehavior mod-els(mathematicalmodelsofdecisionmaking),bymeansof com-biningrecentinsightsfromthebehavioralsciencesandadvances ineconometrictechniques.Hisworkhasreceivedvarious interna-tionalprizes,scholarshipsandpersonalresearchgrants(including recentlya2millioneuroConsolidatorgrantfromtheEuropean Re-searchCouncil).HehaspioneeredtheRandomRegret Minimiza-tionapproachtodiscretechoicemodeling,whichhasbeen incor-poratedinvariouseconometricssoftwarepackages,courses,and textbooksworldwide.

Secure or usable computers? Revealing employees’ perceptions and trade-offs by means of a discrete choice experiment