An asynchronous distributed and scalable generalized Nash equilibrium seeking algorithm for strongly monotone games

Delft University of Technology

An asynchronous distributed and scalable generalized Nash equilibrium seeking algorithm

for strongly monotone games

Cenedese, Carlo; Belgioioso, Giuseppe; Grammatico, Sergio; Cao, Ming

DOI

10.1016/j.ejcon.2020.08.006

Publication date

2021

Document Version

Final published version

Published in

European Journal of Control

Citation (APA)

Cenedese, C., Belgioioso, G., Grammatico, S., & Cao, M. (2021). An asynchronous distributed and scalable

generalized Nash equilibrium seeking algorithm for strongly monotone games. European Journal of Control,

58, 143-151. https://doi.org/10.1016/j.ejcon.2020.08.006

Important note

To cite this publication, please use the final published version (if applicable).

Please check the document version above.

Copyright

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons. Takedown policy

Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim.

This work is downloaded from Delft University of Technology.

ContentslistsavailableatScienceDirect

European

Journal

of

Control

journalhomepage:www.elsevier.com/locate/ejcon

An

asynchronous

distributed

and

scalable

generalized

Nash

equilibrium

seeking

algorithm

for

strongly

monotone

games

Carlo

Cenedese

a,∗

_,

_Giuseppe

_Belgioioso

b

_,

_Sergio

_Grammatico

c

_,

_Ming

_Cao

a

a Engineering and Technology Institute Groningen (ENTEG), University of Groningen, the Netherlands b Control System group, TU Eindhoven, Eindhoven, the Netherlands

c Delft Center for Systems and Control, TU Delft, the Netherlands

a

r

t

i

c

l

e

i

n

f

o

Article history: Received 12 March 2020 Revised 3 August 2020 Accepted 8 August 2020 Available online 25 August 2020 Recommended by Prof. T. Parisini Keywords: Game theory Variational GNE Monotone games Asynchronous update Delayed communication Operator theory

a

b

s

t

r

a

c

t

Inthispaper,wepresentthreedistributedalgorithmstosolveaclassofGeneralizedNashEquilibrium (GNE)seekingproblemsinstronglymonotonegames.Thefirstone(SD-GENO)isbasedonsynchronous updatesoftheagents,whilethesecondandthethird(AD-GEEDandAD-GENO)representasynchronous solutionsthatarerobusttocommunicationdelays.AD-GENOcanbeseenasarefinementofAD-GEED, sinceitonlyrequiresnodeauxiliaryvariables,enhancingthescalabilityofthealgorithm.Ourmain contri-butionistoproveconvergencetoav-GNEvariational-GNE(vGNE)ofthegameviaanoperator-theoretic approach.Finally,weapplythealgorithmstonetworkCournotgamesandshowhowdifferentactivation sequencesanddelaysaffectconvergence.Wealsocomparetheproposedalgorithmstoastate-of-the-art algorithmsolvingasimilarproblem,andobservethatAD-GENOoutperformsit.

© 2020TheAuthors.PublishedbyElsevierLtdonbehalfofEuropeanControlAssociation. ThisisanopenaccessarticleundertheCCBYlicense.(http://creativecommons.org/licenses/by/4.0/)

1. Introduction

In modern society, multi-agent network systems arise in sev-eral areas, leading to increasing research activities. When self-interested agents interact between each other, one of the best mathematical tools to study the emerging collective behavior is noncooperative game theory over networks. In fact, networked gamesemergesinseveralapplicationdomains,suchassmartgrids [8,12], social networks [10,19,20] and distributed robotics [6]. In a game setup, the players(or agents) aim at minimizing a local andprivatecostfunctionwhichrepresentstheirindividualinterest, and,atthesametime,satisfylocalandglobalconstraints,limiting thepossibledecisions(orstrategies/actions).Thecostfunctionand constraints ofa singleplayerare influenced by thebehavior ofa fractionoftheothers,called“neighbors”.Thus,eachdecisionis in-fluenced by some localinformation,which istypically exchanged withtheneighbors.Onepopularnotionofsolutionforthesegames istheGNE,wherenoplayerbenefitsfromunilaterallychangingits strategy,see[16].

In [3,20,30], the authors focused on developing synchronous and distributed equilibrium seeking algorithms for

noncoopera-∗ _{Corresponding author.}

E-mail address: c.cenedese@rug.nl (C. Cenedese).

tivegames, namely,thecaseinwhichall theagentsupdatetheir strategiesatthesametime. Eventhough thisassumption isquite common,itmayleadtoseverlimitationsinthecaseofagentswith heterogeneouscomputational capabilitiesin thegame. For exam-ple,consideranallocationgamebetweenseveralprocessors,asin [31],andassume thattheyareoftwotypes:highandlow perfor-mances.Asynchronousupdateschemeimpliesthatalltheplayers mustcomplete their current update, beforea new one can start. Thus, the low performance processors create a bottleneck in the overall performance. Toovercome thisproblem, we focuson de-velopingasynchronousupdaterules.Infact,itisknownthat asyn-chronicity can speed up the convergence, facilitate the insertion ofnewagentsinthenetwork andevenincrease robustnessw.r.t. communicationfaults,see[5]andreferencestherein.

Among thevery firstworkson asynchronous distributed opti-mization,theoneofBertsekasandTsitsiklisin[4]standsout.From thereonward, severalauthorselaboratedontheseideasand pro-ducednovelresults forconvexoptimization[11,23,27].In [31],Yi andPaveldevelopedanasynchronousalgorithmtosolve noncoop-erativegeneralizedgamessubjecttoequalitycouplingconstraints. Thisresultwasenabledbytheframework(ARock),recently intro-ducedbyPengetal.in[26],that providesa widerange of asyn-chronousvariationsoftheclassicalfixedpointiterativealgorithms. In this paper, we propose an asynchronous algorithm robust to delayedinformation to solve noncooperative games subject to https://doi.org/10.1016/j.ejcon.2020.08.006

0947-3580/© 2020 The Authors. Published by Elsevier Ltd on behalf of European Control Association. This is an open access article under the CC BY license. ( http://creativecommons.org/licenses/by/4.0/ )

144 C. Cenedese, G. Belgioioso and S. Grammatico et al. / European Journal of Control 58 (2021) 143–151

affinecouplingconstraints.Thisworkgeneralizesandextendsthe currentliteratureonthetopicinthefollowingways.

• We tackle the case of a game subject to inequality coupling constraint (rather than only equality constraint). This drasti-cally broaden the type ofproblemsthat can be solved by the proposed approach.Forexample insignal processing[29] and smartgrids [12]inequality constraintsarise naturally.This ex-tensioncannotbeachievedviaanextension oftheresults cur-rently available dueto the different control structure consid-ered.

• The algorithms that we develop rely on node variables only, ratherthanedgevariablesasin[31].This,apparentlysubtle dif-ference,leadstoasolutionthatadopts(almostalways)alower numberofvariables.So,itislighterfromacomputationalpoint ofview,requireslessmemoryandinvolveslighter communica-tionbetweenagents.Allthesefeaturesmaketheproposed so-lutionachieveoverallbetterperformancesthan[31].

We concludethepapercomparing theproposed algorithmsto thatin[31],forthecaseofaCournotgame,showingthatour al-gorithmsachievefasterconvergence.Apreliminaryandpartial ver-sionoftheseresultswerepresentedin[7].

2. Notation

2.1.Basicnotation

The set of real, positive, and non-negative numbers are de-notedby_R,R>0,R≥0,respectively;R:=R∪

{∞}

.Thesetof natu-ralnumbersisN.ForasquarematrixA∈Rn×n_{,itstransposeisA}_, [A]iis theith rowofthematrix and[A]ij representsthe element

inithrowandjthcolumn.A_0(A ₀₎standsforapositivedefinite (semidefinite)matrix.ABistheKroneckerproductofthe matri-cesAandB.TheidentitymatrixisdenotedbyIn∈Rn×n.0(resp.1)

isthevector/matrixwithonly0(resp.1)elements.Forx1,...,xN∈

Rn_{, the collective vector is denoted by x:=col}

₍₍

_x

i

)

i∈(1,...,N)

)

:= [x₁,. . .,x_N]. diag

((

Ai

)

i∈(1,...,N)

)

describesa block-diagonal matrix withA1,...,AN onthemaindiagonal.

2.2.Operator-theoreticnotation

The identityoperatorisdenotedbyId(· ).Thesetvalued map-pingN_C:_Rn_{⇒ R}n _{denotesthenormalconetothesetC⊆ R}n_,that

isN_C

(

x

)

=

{

u∈Rn

_|

_sup



_{C− x,u}



_{≤ 0}

_}

_{ifx∈C and∅otherwise.The}

graph of a set valued mapping A:X⇒ Y is gra

(

A

)

:=

{

(

x,u

)

∈

X× Y

|

u∈A

(

x

)

}

.TheprojectionoperatoroveraclosedsetS⊆ Rn

isproj_S

(

x

)

:Rn_{→S andit isdefinedasproj}

S

(

x

)

:=argminy∈S



y−

x



2_{.A set valued mappingF:R}n_{⇒ R}n _{is -Lipschitz continuous}

with>0,if



u−

v



≤ 



x− y



forall

(

x,u

)

,

(

y,

v

)

∈gra

(

F

)

;Fis (strictly) monotone if forall

(

x,u

)

,

(

y,

v

)

∈gra

(

F

)



u−

v

,x− y



≥

(

>

)

0 holds true, and maximally monotone if it does not exist a monotone operator witha graph that strictly containsgra

(

_F

)

. Moreover,itis

α

-stronglymonotoneifforall

(

x,u

)

,

(

y,

v

)

∈gra

(

F

)

it holds



x− y,u−

v



≥

α

x− y



2_{. The operator F is}

_η

_-averaged (

η

-AVG) with

η

_∈(0, 1) if



_F

(

x

)

_{− F}

(

y

)



2_≤

_

_{x− y}

_

2₋1−η

η



(

Id−

F

)(

x

)

−

(

Id− F

)(

y

)



2 _{forallx,y∈R}n_{; F is}

_β

_{-cocoerciveif}

_β

_{F is}

1

2-averaged,i.e.firmlynonexpansive(FNE).Theresolventofan op-eratorA:Rn_{⇒ R}n_isJ

A:=

(

Id+A

)

−1.

3. Problem formulation

3.1.Mathematicalformulation

We consider a noncooperative game



between N agents (or players)subjecttoaffinecouplingconstraints.Wedefinethegame

as the triplet



:=

(

X,

{

fi

}

i∈{1...N},G

)

, where its elements are

re-spectively: the collective feasible decision set, the players’ local cost functions andthe graph describing the communication net-work.Inthefollowingsubsections,eachoneofthemisintroduced.

3.1.1. Feasiblestrategyset

Everyagenti_∈N:₌

{

1_,...,N

}

hasalocaldecisionvariable(or strategy)xibelongingto its privatedecisionset



i⊂ Rni,namely

the set of all those strategies that satisfy the local constraints of player i. The collective vector of all the strategies, or strat-egy profile of the game, is denoted as x:=col

(

x1,...,xN

)

∈Rn,

wheren:=i∈Nni.Then,allthedecisionvariablesofallthe

play-ersother than i are representedvia thecompact notationx_−i:= col

(

x1,...,xi−1,xi+1,...,xN

)

.We assume that theagents are

sub-jecttom affinecouplingconstraintsdescribedby theaffine func-tion x→Ax+b, whereA∈Rm×n _andb∈Rm_{. Thus, thecollective}

feasibledecisionsetcanbewrittenas

X :=



∩

{

x ∈Rn

_|

_{Ax ≤ b}

_}

_{, (1)}

where



=i∈N



i⊂ Rn,istheCartesianproductofthelocal

con-straintssets



i’s.Accordingly,thesetofall thefeasiblestrategies

ofeachagenti_∈N readsas Xi

(

x −i

)

:=



y∈



i

|

Aiy− bi≤  j∈N\{i}



bj− Ajxj





,

whereA₌[A1,...,AN], Ai∈Rm×ni andNj=1bj=b. The choice of

affine coupling constraints is widely spread in the literature of noncooperative games, see e.g., [10,24,30]. Moreover, in[20], Re-mark3,itishighlighted thatseparableandconvexcoupling con-straintscan always berewritten inanaffine form. Finally,we in-troducesome blanketassumptionsonthissetoffeasiblestrategy, standardintheliterature[9,10,16,30,31].

Standing Assumption 1 (Convexconstraintsets). Foreach player

i∈N,theset



iisconvex,nonemptyandcompact.Thecollective

feasibleset_X satisfiesSlater’sconstraintqualification.

3.1.2. Costfunctions

Each player i_∈N has a local cost function f_i

(

x_i,x_−i

)

:



_i×



−i→R, where



−i:=j∈N\{i}



j. The coupling between the

players appears not only in the constraints but also in the cost function,duetothedependencyonboth xiandx−i.Next,we as-sumesomepropertiesforthesefunctionsthatareextensivelyused intheliterature[16,30].

Standing Assumption 2 (Convex and differentiable cost func-tions). For all i∈N, the cost function fi

(

xi,x−i

)

is continuously differentiableandconvexinxi.

3.1.3. Communicationnetwork

The communication betweenagents is described by an undi-rectedandconnectedgraphG=

(

N,E

)

whereE⊆ N× N istheset ofedges.Giventwoagentsi_,j_∈N,thecouple(i,j)belongsto_E, ifagentisharesinformationwithagentjandviceversa.Thenwe saythatjisaneighbourofi,i.e., j∈NiwhereNiisthe

neighbour-hoodofi.ThenumberofedgesinthegraphisdenotedbyE:₌

|

_E

|

. TodefinetheincidencematrixV∈RE×N _{associatedtoG,letus} la-beltheedges asel,forl∈

{

1,...,E

}

.Wedefinetheentry[V]li:=1

(resp.₋₁) ife_l=

(

i_,·

)

(resp.e_l=

(

_·,i

)

) and0otherwise.The deci-sionofwhichofthetwoagentscomposinganedgeisthesinkand whichthesourceisarbitrary.Byconstruction,V1 N=0 N.Then,we

define Eout

i (resp.Eiin) asthesetof allthe indexesl ofthe edges

el that start from(resp.endin) node i,andhenceEi=Eiout∪Eiin.

ThenodeLaplacianL∈RN×N_{ofanundirectedgraphisasymmetric} matrixdefinedbyL:=VV.AnotherimportantpropertyofL,used intheremainder,isL1 N=0 N.

3.2. GeneralizedNashequilibrium

In summary,the considered generalizedgame is describedby thefollowingsetofinter-dependentoptimizationproblems:

∀

i∈N :



argmin y∈Rni fi

(

y,x −i

)

s.t. y∈Xi

(

x −i

)

. (2) The most popular equilibrium concept considered for nonco-operativegames withcouplingconstraintsisthe generalizedNash equilibrium, thus the configuration in which all the relations in (2)simultaneouslyhold.

Definition 1 (GeneralizedNash Equilibrium). Acollectivestrategy

x∗∈X isageneralizedNashequilibrium(GNE)if,foreachplayer

i,itholds fi

(

x∗i,x ∗−i

)

≤ inf

fi

(

y,x ∗_−i

)

|

y∈Xi

(

x ∗_−i

)

.

Inthiswork, wefocusonasubset ofGNE,thesocalled varia-tionalGNE(vGNE),aclassofequilibriathatisconsideredinmany otherworksthroughouttheliterature– see[3,16,21,22].Thename of theseequilibriaderivesfromthe fact that they can be formu-lated as the solutions to a variational inequality (VI). An impor-tantpropertyoftheseequilibriaisthateachagentfaces thesame penaltytofulfillthecouplingconstraints,whichisparticularly use-ful to represent a “fair” competition between agents [16]. Varia-tionalGNEcanbeseenasaparticularcaseoftheconceptof nor-malized equilibriumpoints,firstly introducedby Rosenin[28] and furtherstudiedin[10,25].

Toproperlycharacterizethisset,wedefinethepseudo-gradient

mapping(orgamemapping)of(2),as

F

(

x

)

=col

(

(

∇

xifi

(

xi,x −i

))

i∈N

)

. (3)

The pseudo-gradientgathers in a collective vector form the gra-dients of the cost functions each w.r.t. the local decision vari-able.Next,weintroducesomestandardtechnicalassumptions,e.g., [2,13].

Standing Assumption 3. ThemappingFin(3)is

α

-strongly mono-toneand-Lipschitzcontinuous,forsome

α

,>0.

WhenStandingassumption2holdstrue,themappingFis sin-gle valuedandthesetofvGNE ofthegamein(2)correspondsto the solution to VI(F,X), namelythe problem of finding a vector

x∗_∈X such that



F

(

x ∗

)

,x − x∗



_{≥ 0,}

_∀

_{x ∈X . (4)}

The continuity of F (Assumption 2) and compactness of X

(Assumption1)implytheexistenceofasolutiontoVI(F,X),while the strong monotonicity (Assumption 3) entails uniqueness, see [15],Th.2.3.3.

Next,let usdefine theKKT conditions associatedto thegame in (2).The strong duality ofthe problem(Assumptions 1 and2) impliesthat,ifx∗isaGNEof(2),thenthereexistNdualvariables

λ

∗

i ∈Rm≥0,foralli∈N, suchthatthefollowing inclusionsare sat-isfied:

∀

i∈N :



0∈

∇

xifi

(

x∗i,x∗−i

)

+Ai λ∗i +Ni

(

x∗i

)

, 0∈b− Ax ∗+N_Rm ≥0

(

λ

∗ i

)

. (5) Instead of looking for the solution of the general case where

λ

∗

1,...,

λ

∗N may be different, we examine the special case when

λ

∗_:=

_λ

∗ 1=· · · =

λ

∗N,namely

∀

i∈N :



0∈

∇

xifi

(

x∗i,x∗−i

)

+Ai

λ

∗+Ni

(

x∗i

)

, 0∈b− Ax ∗+N_Rm ≥0

(

λ

∗

₎

_. (6)

Itfollowsfrom[17],Th.3.1(ii),thattheKKTinclusionsin(6) cor-respondtothesolutionset toVI(F,X).Thus, everysolutionx∗to

VI(F,X)isalsoaGNEofthegamein(2),[17,Th.3.1(i)].Sincethe solutionsettoVI(F,X)isasingleton,weconcludethatthereexists auniquevGNEofthegame(2).

4. Synchronous distributed GNE seeking algorithm

We first introduce the synchronous counterpart of AD-GENO, i.e., the Synchronous Distributed GNE Seeking Algorithm with Node variables (SD-GENO).The derivation of thealgorithm is based on an operator splitting approachto solve the KKT system in(6). A similarapproachwasalsoadoptedin[3,30]inthecontestofGNE findingproblems.

4.1. Algorithmdesign

The KKT conditions of each agent i in (5) are satisfied by a couple (x_i,

λ

_i), where the dual variables

λ

_i may be differ-ent among the players. If we enforce the consensus among the dual variables, then the unique solution of the inclusions is the vGNE of the game. This is achieved by exploiting the fact that ker

(

V

)

=span

(

1

)

and introducing the auxiliary vari-ables

σ

l,l∈

{

1,...,E

}

, one for every edge in the graph.

Us-ing the notations

λ

:₌col

((

λ

_i

)

_i∈N

)

_∈RmN_,

_:=diag

₍₍

_A i

)

i∈N

)

∈ RmN×n_{, ¯b:=col}

₍₍

_bi

₎

_i

∈N

)

∈RmN,

σ

:=col

((

σ

l

)

l∈{1...E}

)

∈RmE, V:=

V Im∈RmE×mNandL:=L Im∈RmE×mN,wecasttheaugmented

versionoftheinclusionsin(5)by 0 ∈F

(

x

)

+



_λ

_+N

₍

_x

₎

0 ∈¯b−

x +N_RmN ≥0

(

λ

)

+L

λ

+

ρ

V 

_σ

0 =−

ρ

V

λ

, (7) where

ρ

∈R>0.In(7),thetermL

λ

acceleratestheconvergenceof thedualvariablestoconsensus.

A solution



₌col

(

x∗_,

σ

∗_,

λ

∗

)

of the above inclusionscan be equivalentlyrecastasazeroofthesumoftwomappingsAandB

definedas A:



→



_{0 0}

_ 0 0 −

ρ

V −

ρ

V  0



+



_N 

(

x

)

0 N_RmN ≥0

(

λ

)

B:



→



_F

₍

_x

₎

0 ¯b+L

λ

. (8)

Infact,



∗∈zer

(

A+B

)

ifandonlyifϖ∗_{satisfies(7).}

Next,weshowthatthezerosofA+B characterizethevGNEof theoriginalgame.

Proposition 1. Let_Aand_{B be}asin(8).Thenthefollowinghold:

(i) zer

(

A+B

)

=∅,

(ii) ifcol

(

x∗_,

σ

∗_,

λ

∗

)

_∈zer

(

_A+B

)

then(x∗,

λ

∗)satisfiestheKKT con-ditionsin(5),with

λ

∗₁=· · · =

λ

∗

N,hencex∗istheuniquevGNEof

thegamein(2).

Theproof isattainedby exploitingthepropertythat ker

(

V

)

= ker

(

L

)

,forthegraphdescribedinSection3.1.3.Thestepsare sim-ilartothosein[30,Th.2].Weomitthemhereforbrevityreasons. Severalresearchershaveanalyzedtheproblemoffindingazero of the sum of two monotone operators. The so called splitting

methodsrepresentoneofthemostpopularapproachdevelopedto

attain an iterative algorithm to solve this class ofproblem - see [14],[1,Ch.26].

Lemma 1. The mappings AandB in (8)are maximallymonotone. Moreover,B is

χ

-cocoercive,where

χ

:=min

α

2,

λ

max

(

L

)

−1

.

Thepropertiesoftheoperatorsprovedabovedriveustoselect thepreconditionedforward-backwardsplitting(PFB)toderivea dis-tributedanditerativealgorithmseekingzer

(

A+B

)

.Thisapproach waspreviouslyadoptedbyotherresearchers,e.g.,[30].

146 C. Cenedese, G. Belgioioso and S. Grammatico et al. / European Journal of Control 58 (2021) 143–151

ThePFBsplittingoperatorreadsas

T:=J_−1_A◦

(

Id−



−1B). (9)

Theso-calledpreconditioningmatrix



isdefinedby



:=



_τ

_{−1 0 −}

_ 0

δ

−1_ImM

_ρ

_V −

ρ

V 

ε

−1

(10) where

δ

_∈R>0,

ε

=diag

((

ε

i

)

i∈N

)

 Imwith

ε

i>0,foralli∈N and

τ

isdefinedinasimilarway.

Theupdateruleofthealgorithmisobtainedbyincludinga re-laxationstep,i.e.,

˜



(

k

)

=T



(

k

)



(

k+1

)

=



(

k

)

+

η

(



˜

(

k

)

−



(

k

))

. (11) It comes from (9) that fix

(

T

)

=zer

(

A+B

)

, in fact



∈ fix

(

T

)

⇔



∈T



⇔0∈



−1

₍

_A+B

₎



_⇔



_∈zer

₍

_A+B

₎

_{, see} [1,Th.26.14].

In the remainder of this section, we provide the complete derivation of SD-GENO, obtained directly from (11). In the fol-lowing,we denoteϖ :=ϖ(k),



+:=



(

k+1

)

and



˜ :=



˜

(

k

)

to simplify the notation. Consider



˜ ₌T



. From (9) it holds that

˜



=J_−1_A◦

(

Id−



−1B

)



⇔

(



− ˜



)

∈A



˜ +B



,thus

0∈A



˜ +B



+

(



˜ −



)

. (12)

The update rule of each components ofϖ is attained by analyz-ing the rowblocks of (12). The first readsas 0 _∈F

(

x

)

₊N_

(

x˜

)

₊

τ

−1

₍

_{x˜− x}

₎

₊



_λ

_{.Bysolvingthisinclusionby ˜ x,weattainthe} up-daterulefortheprimalvariables:

˜

x =JN◦



x −

τ

(

F

(

x

)

+



_λ

₎



_{. (13)} Similarly,fromthesecond rowblockof(12),weattaintheupdate for ˜

σ,

i.e.,

˜

σ

=

σ

+

δρ

V

λ

. (14)

Finally, the third row block of (12) is 0 ∈¯b+L

λ

+N_RmN

≥0

(

˜

λ

)

+

(

2x˜− x

)

+

ρ

V

(

2

σ

˜−

σ

)

+

ε

−1

₍

_λ

˜₋

_λ

₎

_{,fromwhichweobtain} ˜

λ

=JN_RmN ≥0 ◦



λ

+

ε

((

2x ˜− x

)

− ¯b−

ρ

V 

(

2

σ

˜−

σ

)

− L

λ

)



(14) = proj_RmN ≥0



λ

+

ε

((

2x ˜− x

)

− ¯b −

ρ

V 

σ

−

(

2

δρ

2₊₁

₎

_L

_λ

₎



_{. (15)} Note that, theupdate of

λ

˜ dependsonly onthe aggregate in-formation V

σ

. We can exploit this feature to replace the edge auxiliaryvariables

σ

l’s,withasinglevariableforeachagenti

de-finedbyzi:=



[V]i Im



σ

∈RmN.RecallingthatVV=L Im=:

L,wecomputetheupdateruleofthesenewvariablesandreplace (14)by

˜

z =V 

σ

+

δρ

V V

λ

=z +

δρ

L

λ

. (16) Consequently,(15)ismodifiedaccordinglyas

˜

λ

=proj_RmN ≥0



λ

+

ε

((

2x ˜− x

)

− ¯b −

ρ

z −

(

2

δρ

2₊₁

₎

_L

_λ

₎



_{. (17)}

Toensurethatthischangeofvariablesdoesnotaffectthe equilib-riumofthegame,we introducethefollowing resultprovingthat anequilibriumpointofthenewsetofequationsisindeedavGNE of(2).

Theorem 1. Ifcol(x∗,z∗,

λ

∗)isasolutiontotheEqs.(13),(16),(17), with 1 z∗₌0_,thenx∗isavGNEofthegamein(2).

Remark 1. In[31],thealgorithmSYDNEYachievesconvergenceto thevGNEofthegame(2),whenthisissubjecttoequalitycoupling

constraintsonly.Thissolutionreliesonedgeauxiliaryvariablesto enforce theconsensus of the

λ

i’s.Therefore, thenumber of

vari-ablesthateachagenthastostoreisO

(

N

)

.

Thechangeof“variables”,from

σ

toz,isconvenientwhenthe edges outnumber the nodes,which isalmost always the case. In fact,alower numberofvariablesleadstoanoverall incrementin thealgorithmic efficiencyandto afixed memoryrequirementfor eachplayerthatdoesnotincreasewithN.Furthermore,ifSYDNEY in[31] ismodifiedto addressinequality constraints,it would re-quirean additionalround ofcommunication betweentheagents, makingitmoredemandingandslowerthanSD-GENO.

4.2. Synchronous,distributedalgorithmwithnodevariables (SD-GENO)

Thecompleteformulationofthealgorithmisobtainedby gath-eringtogetheralltheupdaterulesintroducedintheprevious sec-tion,i.e.,(13),(16),(17)andaddingarelaxationstep.Thealgorithm incompactformisexpressedas

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

˜ x =proj_



x −

τ

(

F

(

x

)

+



_λ

₎



˜ z =z +

ρδ

L

λ

˜

λ

=proj_RmN ≥0



λ

+

ε

((

2x ˜− x

)

− ¯b −

ρ

z −

(

2

δρ

2₊₁

₎

_L

λ

₎



x += x +

η

(

x ˜− x

)

z += z +

η

(

z ˜− z

)

λ

+₌

_λ

₊

_η

₍

_λ

_˜−

_λ

₎

_, (18)

whilethe localupdatesandthe initial conditionofSD-GENO are provided in Algorithm 1. It is composed of two main phases:

Algorithm 1: SD-GENO.

Input: k₌0,foralli_∈N,x_i

(

0

)

_∈Rn_i,

_λ

i

(

0

)

∈Rm,zi

(

0

)

= 0 m.

Choose

δ

,

ε

i,

τ

isatisfying(19),while

η

∈

(

0,1

)

and

ρ

∈

(

0,1].

Iteration k:

Communication: eachi∈N gathers

λ

j

(

k

)

fromthe

neighborsandupdatesthedisagreementvector

d_i

(

k

)

:₌_j∈N

i

(

λ

i−

λ

j

)

Local update, for i∈N do

˜ x_i=proj_ i



x_i−

τ

_i

(

∇

_if_i

(

x_i,x_−i

)

₊A_i

λ

_i

)



˜ zi=zi+

ρ δ

di

(

k

)

˜

λ

i=projRm ≥0



λ

i+

ε

i

(

Ai

(

2x˜i− xi

)

− bi −

ρ

zi−

(

2

δρ

2+1

)

di

(

k

)



x+_{i =}x_i+

η

(

x˜_{i− xi}

)

z+_i =zi+

η

(

z˜i− zi

)

λ

+ i =

λ

i+

η

(

˜

λ

i−

λ

i

)

k←k+1

thecommunicationwiththeneighborsandthelocalupdate.First, each agentgathers the information about the strategies and the dual variables of the neighbors. Next, the local update is per-formed, based on a gradient descent and dual ascend structure. It isworth noticingthat onlyone roundof communicationis re-quiredateachiterationofSD-GENO.

TheconvergenceofSD-GENOtothevGNEofthegamein(2)is proveninthefollowingtheorem.

Theorem 2. Setthestepsizes

ε

i,

δ

,

τ

i,foralli∈N,and

ϑ

∈Rsuch

that

τ

i≤

(



Ai



+

ϑ

)

−1 (19a)

ε

i≤

(

ρ|

Ni

|

+



Ai



+

ϑ

)

−1, (19c)

ϑ

> 1

2

χ

(19d)

with

χ

as in Lemma 1and

η

∈



0,4χϑ−1 2χϑ



.Then, the sequence

(

x

(

k

))

_k∈NgeneratedbySD-GENO(Algorithm1)convergestothevGNE ofthegamein(2).

5. Asynchronous distributed algorithm with edge variables (AD-GEED)

In the case of heterogeneous agents with very different up-date rates,SD-GENO canconvergeslowly, duetoits synchronous structure.Toovercomethislimitation,weintroduceherethe Asyn-chronousDistributedGNESeekingAlgorithmwithEdgevariables (AD-GEED). It uses edge auxiliary variables

{

σ

l

}

l∈{1...E} and an

asyn-chronousupdatetocomputethevGNEofthegamein(2).As dis-cussed in the previous section, the use of edge-based auxiliary variables may lead to a limitedscalability ofthe final algorithm. InSection6,weuseAD-GEEDasastartingpointtodevelopan al-gorithm relying onnode variablesonly. Froma technicalpointof view,theasynchronicityisachievedbyexploitinganasynchronous framework forfixed-point iterations,thesocalled“ARock” frame-work,developedin[26].

5.1. Algorithmdesign

The update rule in the asynchronous case, is similar to that in(11),withthe maindifference that,ateach iteration, onlyone agenti∈Nupdatesitsstrategyxi,dualvariable

λ

iandlocal

auxil-iaryvariables

{

σ

l

}

_l∈Eout

i

.Tomathematicallyformulatethisconcept, we introduce N diagonalmatricesHi,where[Hi]jj is1 ifthej-th

element of col(x,

σ

,

λ

) is an element of col

(

xi,

{

σ

l

}

l∈Eout

i ,

λ

i

)

and

0otherwise. ThematrixHi triggersthe updateofthose elements

inϖ that areassociated toagenti.We assumethat thechoice of whichagentperformstheupdateduringtheiterationkisruledby ani.i.d.randomvariable

ζ

(k),takingvaluesinH:=

{

Hi

}

i∈N.Given a discrete probability distribution

(

p1,...,pN

)

, letP[

ζ

(

k

)

=Hi]=

p_i, for all i_∈N. Therefore, the update rule in the asynchronous caseiscastas



(

k+1

)

=



(

k

)

+

ηζ

(

k

)(

T



(

k

)

−



(

k

))

. (20) Anillustrativeexampleisnowprovided toclarifyhowto con-structthesetH.

Example 1. ConsideragamewithN=3,E=2,m=1,ni=1,i=

1,2,3andϖ isthecollective vectorofall thestrategiesand aux-iliary variables in the game. The communication network is de-scribedbytheundirectedgraph_G,wherethearrowsdescribethe conventionadoptedfortheedges.

Inthiscase,Hisasetofthree8× 8matrices,namely

H 1 :=diag

((

1,0,0,1,0,1,0,0

))

H 2 :=diag

((

0,1,0,0,1,0,1,0

))

H 3 :=diag

((

0,0,1,0,0,0,0,1

))

.

Ifduringiterationkagent2isupdating,(20)turnsinto



(

k+1

)

=



(

k

)

+

η

H 2

(

T



(

k

)

−



(

k

))

. (21) So,theonly elementsof ϖ thatchange are(x2,

σ

2,

λ

2), precisely thevariablesassociatedtoagent2.

Weassume thateachagenti isequippedwithpublicand pri-vatememory, theformer isused by theneighbors towrite their strategies (and dual/auxiliary variables) when they complete an update. The latter instead is used by i to store a copy of the publicmemory, whenit isperforming alocal update.This mem-ory is not accessible to the neighbors, so it ensures the consis-tency of the local updates, refer also to [26]. If an agent j∈Ni

concludes its update while agent i is still computing its future strategy during iteration k, then the value of the strategy of j, which agent i is using, becomes outdated. We denote the vec-tor of possibly outdated strategy used for the update during it-eration k as



ˆ

(

k

)

. All the variables updated by an agent i, i.e.,

x_i,

λ

_i and

{

σ

_l

}

_l∈Eout

i , share the same delay

ϕ

i

(

k

)

∈N, since they

are written at the same moment in the neighbors’public mem-oriesofitsneighbors.Technically,thecomponentsof



ˆ

(

k

)

associ-atedtoagentj=iusedduringthek-thiterationbyagentiforthe updatearecol

(

xj

(

k−

ϕ

j

(

k

))

,

{

σ



(

k−

ϕ

j

(

k

))

}

_∈Eout

j

,

λ

j

(

k−

ϕ

j

(

k

)))

,

hence



ˆj

(

k

)

=



j

(

k−

ϕ

j

(

k

))

.

According to this, the final formulation of the update rule (20)becomes



(

k+1

)

=



(

k

)

+

ηζ

(

k

)(

T− Id

)



ˆ

(

k

)

. (22) Theonlyassumptionthatweimposeoverthedelay,is bound-edness,asformalizednext.

Assumption 4 (Bounded maximum delay). The delays are uniformly upper bounded, i.e. there exists

ϕ

¯>0 such that sup_k∈N

≥0maxi∈N

{

ϕ

i

(

k

)

}

≤ ¯

ϕ

<+∞.

The local update rules of AD-GEED are presented in Algorithm2 andthey areachievedvia steps similarto those in-troducedinSection4.1forSD-GENO.Toeasethenotation,foreach agent j∈N, we define xˆj:=xj

(

k+

ϕ

j

(

k

))

,

λ

ˆj:=

λ

j

(

k−

ϕ

j

(

k

))

and

σ

ˆ_l:₌

σ

_l

(

k₋

ϕ

j

(

k

))

, for all l∈Eout_j , and furthermore

ˆ

x:=col

((

xˆj

)

j∈N

)

, ˆ

λ

:=col

((

λ

ˆj

)

j∈N

)

,

σ

ˆ :=col

((

σ

ˆj

)

j∈N

)

. Notice thateach agenthasalwaysaccessto themostrecentvalueofits variables,i.e.,

ϕ

i

(

k

)

=0foreveryagenti∈N.

Thefollowingconvergencetheoremisachivedbyexploitingthe resultsin[26]foraKrasnosel’ski˘ı asynchronousiteration.

Theorem 3. For every i∈N, choose

ε

i,

δ

,

τ

i as in (19), and

let

η

∈



0, cN pmin 2ϕ¯√pmin+1



2− 1 2χϑ



and c∈(0,1).Then, thesequence

(

x

(

k

))

_k∈NgeneratedbyAD-GEED(Algorithm2)convergestothevGNE ofthegamein(2)almostsurely.

Remark 2. Iftheprobabilitydistributionisuniform,i.e., pmin=N1,

andwechoose

ϑ

₌ 1

χ,thentheboundsontherelaxationstep sim-plifyas

η

∈



0,3 2 c √ N 2ϕ¯+√N



.Moreover,ifthereisnodelay,so

ϕ

¯=0,

orthenumberofagentsisveryhigh, theboundsmaybechosen independentlyfromthenumberofplayers,e.g.,as

η

∈(0,1].

ThestructureofAD-GEEDissimilartothatofADAGNESin[31, Algorithm1], where edge auxiliary variables are used to achieve consensusover thedualvariables.However,unlike ADAGNES,our algorithmcanhandleinequality couplingconstraints.Moreover, it hasbetterperformances,intermsofconvergence time,according toournumericalexperience,seeFigure3.

148 C. Cenedese, G. Belgioioso and S. Grammatico et al. / European Journal of Control 58 (2021) 143–151 Algorithm 2: AD-GEED. Input: k=0,x0_∈Rn_,

_λ

0 ∈RmN_,

_σ

0_{= 0} mM,chose

δ

,

ε

i,

τ

i satisfying(?? )and

η

∈

(

0,1

)

.

Iteration k: Selecttheagenti_kwithprobability P[

ζ

(

k

)

=Hi_k]=pi_k

Reading: Agentikcopiesinitsprivatememorythecurrent

valuesofthepublicmemory,i.e.xˆj,

λ

ˆj,

∀

j∈Ni_k and

σ

ˆl,

∀

l∈Ein i_k andl∈Eoutj . Update: ˜ xi_k=proji_k



xi_k−

τ

i_k

(

∇

i_kfi_k

(

xi_k,xˆ−ik

)

+Ai_k

λ

i_k



˜

σ

l=

σ

l+

δρ

(

[V]l Im

)

λ

ˆ,

∀

l∈E_ioutk ˜

λ

i_k=projRm ≥0



λ

i_k+

ε

i_k



Ai_k

(

2x˜i_k− xi_k

)

− bi_k−

ρ

(

[V]i_k Im

)

σ

ˆ −

(

2

δρ

2₊₁

₎

 j∈Ni_k

(

λ

i− ˆ

λ

j

)



x+_i k=xik+

η

(

x˜ik− xik

)

σ

+ l =

σ

l+

η

(

σ

˜l−

σ

l

)

,

∀

l∈E_ioutk

λ

+ i =

λ

i_k+

η

(

λ

˜i_k−

λ

i_k

)

Writing: inthepublicmemoriesofeach j∈Ni_k

(

xi_k,

λ

i_k

)

←

(

x+i_k,

λ

+i_k

)

{

σ

l

}

l∈Eout i_k ←

{

σ

+ l

}

l∈Eout i_k k←k+1

6. Asynchronous, distributed algorithm with node variables (AD-GENO)

Thissection presentsthemainresultofthepaper,namely,we useAD-GEEDasabackbonetodesignan algorithmconvergingin thesamenumberofiteration,butrelying onnode auxiliary vari-ablesonly,andthereforeintrinsicallylighterfromacomputational pointofview.WenameitAsynchronousDistributedGNESeeking Al-gorithmwithNodevariables(AD-GENO).Itisbasedonanideaakin totheoneusedtodevelopSD-GENO.Infact,thelocalupdateof

λ

i

inAD-GEEDrequiresonlytheaggregate quantity([V]iIm)

σ

.We

introduce a variable zi to capturethe variation ofthis aggregate

quantityandshowthatitdoesnotaffectthedynamicsofthepair (xi,

λ

i),thuspreservingtheconvergenceprovedinTheorem3.

Un-likethesynchronouscase,wecannotdirectlydefinez=V

σ,

due tothedifferentupdatefrequenciesof

{

σ

_l

}

_l∈E

i andzithatwould

af-fectthedynamicsof

λ

.Thismismatchisclarifiedviathefollowing example.

Example 2. Consider the communication network in Example 1 and assume that in the first three time instances, agent2 updates twice and then 3 updates once, i.e., i0=i1=2 andi2=3.Fork=1,accordingtoAlgorithm2itholds

σ

2

(

2

)

=

σ

2

(

1

)

+

ηρδ

(

λ

2

(

1

)

−

λ

3

(

0

))

λ

2

(

2

)

∝

ρ

(

σ

2

(

1

)

−

σ

1

(

0

))

, (23) where∝isusedtodescribedependency.Next,fork=2only

λ

3is updated,then

λ

3

(

3

)

∝−

ρσ

2

(

2

)

. (24)

Ifwesubstitutetheedgevariables

σ

1,

σ

2 withzi=[V]i

σ

fori=

1,2,3,andapplythesameactivationsequence,itleadsto

z3

(

3

)

=z3

(

0

)

+

ηρδ

(

λ

3

(

0

)

−

λ

2

(

2

))

λ

3

(

3

)

∝

ρ

z3

(

0

)

. (25) Fromthecomparisonof(24)and(25),itisclearthatthevalueof

λ

3(3)wouldbedifferentinthetwocases.Thisisexplainedbythe factthat

σ

2 isupdatedtwice,whilez3onlyonce.

Tobridgethegapbetween

σ

andz,weintroduceanextra vari-able

μ

_i∈Rm_{foreachnodei.Theroleof}

μ

iistostorethechanges

oftheneighborsdualvariable

λ

j,duringthetimebetweenthelast

updateofiandthenextone.

InAlgorithm3 wepresentthelocalupdaterulesofAD-GENO.

Algorithm 3: AD-GENO.

Input: k=0,x

(

0

)

∈Rn_,

_λ

₍

₀

₎

_∈RmN_,z

₍

₀

₎

₌₀

mN.Forevery

i∈N,choose

δ

,

ε

i,

τ

isatisfying(19),

η

∈

(

0,1

)

andset

μ

i=0 m.

Iteration k: Selecttheagenti_kwithprobability P[

ζ

(

k

)

=Hi_k]=pi_k

Reading: Agentikcopiesitspublicmemoryintheprivate

one,i.e.,thevaluesxˆj,

λ

ˆj,

∀

j∈Ni_k,and

μ

i_k.

Resetthepublicvaluesof

μ

i_k to 0 m.

Update: ˜ xi_k=proji_k



xi_k−

τ

i_k

(

∇

i_kfi_k

(

xi_k,ˆx−ik

)

+A  i_k

λ

i_k

)



˜ zi_k=zi_k+

δημ

i_k ˜

λ

i_k=projRm ≥0



λ

i_k+

ε

i_k

(

Ai_k

(

2x˜i_k− xi_k

)

− bi_k−

ρ

z˜i_k+

(

2

δρ

2− 1

)

j∈Ni_k\{ik}

(

λ

ik− ˆ

λ

j

)



x+_i k=xik+

η

(

x˜ik− xik

)

z+_i k=z˜ik+

ηδρ

 l∈Eout i_k

(

[V]l Im

)

ˆ

λ

λ

+ i_k=

λ

ik+

η

(

˜

λ

ik−

λ

ik

)

Writing: inthepublicmemoryofeach j∈Ni_k

(

x_i k,

λ

ik

)

←

(

x + i_k,

λ

+i_k

)

μ

j←

μ

j+

λ

ˆj−

λ

i_k k←k+1

TheconvergenceofAD-GENOisprovenbythefollowingtheorem. Essentially,weshow thatintroducingzand

μ

doesnotaffectthe dynamicsof(x,

λ

).

Theorem 4. For every i∈N choose

ε

i,

δ

,

τ

i as in (19). Let

η

∈



0, cN p_min 2ϕ¯√p_min+1



2− 1 2χϑ



with pmin:=min

{

pi

}

i∈N and c∈(0, 1).

Then, the sequence

(

x

(

k

))

_k∈N generated by AD-GENO (Algorithm 3) convergestothevGNEofthegamein(2)almostsurely.

Remark 3. Only one extra scalar variable

μ

i is used for every

agent i∈N, and hence the benefits of adopting only node vari-ables, discussed in Remark 1, hold also in this asynchronous counterpart.Furthermore,thenumberofrequiredcommunication roundsbetweenagentsdoesnotincrease,since thevariable

μ

iis

updatedbytheneighborsofagentiduringtheirwritingphase.

7. Simulations

We concludeby proposing two setsof simulations tovalidate thetheoreticalresultsintheprevioussections.First,weapply AD-GENOonanetworkCournot gameandstudyhowdelaysand dif-ferentactivationsequencesaffecttheconvergence.Then,we com-pare the totalcomputationtime required by AD-GENO,AD-GEED

Fig. 1. (a) Action of players { 1 , . . . , 8 } over the three markets A , B , C , D , (b) Com- munication network arising from the competition over the markets.

andADAGNES(in[31,Algorithm1]),overdifferentcommunication graphs.

7.1. AD-GENOconvergence

In anetwork Cournot game,N firmscompete over m markets and the coupling constraints arise from the maximum markets capacities. We consider a smilarformulation to that proposed in [30].Here, weconsidered N=8firms,with thepossibilityto act over m₌3 markets, i.e., x_i∈R3_{, for all i∈N. The local} produc-tionisboundedin0≤ xi≤ xi,whereeachcomponentofxi∈R3is

randomly drawn from[10,45]. In Fig. 1a, the interaction of each firm with the markets is shown, where an edge is drawn be-tween afirmandamarketifone offormer’sstrategiesisapplied to the latter. Two firms are neighbors if they compete over the same market,therefore thecommunicationnetwork betweenthe firms is the one inFig. 1b. The couplingconstraints are defined by Ax≤ b, where A:=[A1,...,AN]∈R3×24 while b∈R3. The

ele-ment [Ai]jk is nonzero, if [xi]k>0 and it is applied to market j.

Each nonzero element inA is randomlychosen from[0.6,1], this value can be seen as the efficiency of a strategy on a market. The componentsof b∈R3 _{are thecapacities ofthemarkets,} ran-domly drawn from[20,100].The localcost function isdefined as

fi

(

xi,x−i

)

:=ci

(

x

)

− P

(

Ax

)

Aixi; ci(x) and it describes the cost of

opting fora certain strategy, while P(Ax) is the reward attained. The priceisassumedlinearin itsargumentP

(

z

)

₌P¯_{− Dz,}where

¯

P∈R3 _{and D∈R}3×3 _{isa diagonalmatrix, their non zero} compo-nentsare randomly chosen from[250,500] and[1,5] respectively. The functionci

(

x

)

=xiQixi+qixiisquadratic,whereQi∈R4×4 is

diagonalandqi∈R4.Theirvaluesarerandomlychosenfrom[1,8]

and[1,4],respectively.

Inordertoexploredifferentsetupswesimulatethreedifferent cases:

(A) Thecommunicationisdelayfree,i.e.,

ϕ

¯=0,andthe activa-tionsequenceisalphabetic,andhenceP[

ζ

(

k

)

=Hi]=N1,for

everyi_∈N.

(B) Theactivationsequenceisstillalphabetic,butthe communi-cationmaybedelayedof3timeinstantsatmost,i.e.,

ϕ

¯=3. (C) Thecommunicationhasnodelay,buttheprobabilityof up-dateis differentbetweenagents,halfofthemhave pi=16, whiletherestpi=121.

The outcome of these scenarios are presented in Fig. 2. The main difference from case (A) can be noticed if there is a non-uniformupdate probability,i.e.,case(C).Infact, wenoticethat a skewer probabilityimpliesslowerconverge.Fromsimulations, we noticedthattheconvergenceofthedualvariablesisoftenthe bot-tleneck to high convergence performances. In all our algorithms, wemitigatedthiseffectbyanappropriatetuningof

ρ

.

7.2. Comparisonbetweenalgorithms

Next, we compare the performance of AD-GENO withrespect to AD-GEED and ADAGNES, from a computational time point of

Fig. 2. (a) Normalized distance from the vGNE, (b) Norm of the disagreement be- tween the dual variables, (c) Constraints violation.

Fig. 3. Comparison of the computation time of ADAGNES vs AD-GENO (orange dia- mond) and AD-GEED vs AD-GENO (blue dots), w.r.t. the variation of the communi- cation network connectivity.

view.Forthe comparisonwithADAGNES,we consideramodified version of the Nash–Cournotgame presented inSection 7.1with couplingequality constraints,i.e., only Ax=b. Here,we consider

N=40firms,eachwithatmostni=2products.Toprovidean

ex-tensive comparison, we considered many instances of this game varyingthecommunicationbetweenagents,fromacompletetoa sparsegraph.The other quantities inthegames are chosen asin theprevioussection.Wecomparedthealgorithmsover160 differ-entgraphs.Thecomputationaltimerequiredtoobtainconvergence iscomparedinthethreecases.1

1 The computation is performed on a single computer, thus the considered time