From Contraction Theory to Fixed Point Algorithms on Riemannian and Non-Euclidean Spaces

(1)

From Contraction Theory to Fixed Point Algorithms on Riemannian and Non-Euclidean Spaces

Francesco Bullo, Pedro Cisneros-Velarde, Alexander Davydov, Saber Jafarpour

Abstract— The design of fixed point algorithms is at the heart of monotone operator theory, convex analysis, and of many modern optimization problems arising in machine learning and control. This tutorial reviews recent advances in understand- ing the relationship between Demidovich conditions, one-sided Lipschitz conditions, and contractivity theorems. We review the standard contraction theory on Euclidean spaces as well as little-known results for Riemannian manifolds. Special emphasis is placed on the setting of non-Euclidean norms and the recently introduced weak pairings for the `1and `∞norms. We highlight recent results on explicit and implicit fixed point schemes for non-Euclidean contracting systems.

I. INTRODUCTION

Motivated by control, optimization, and machine learning applications, this document provides a simplified and incomplete tutorial about the main contraction theorem and resulting fixed point algorithms. The combination of contraction theory and fixed point algorithms originates in the classic ground-breaking paper by Desoer and Haneda [7];

these ideas play a central role in numerical integration of differential equations [8].

The importance of fixed point strategies in modern day data science is described in the recent review [4]. [14] is a recent survey on monotone operators and their application to convex optimization. In this paper, we argue that contraction theory for vector fields is the continuous-time equivalent of these theories. Indeed, strongly monotone operators and gradient vector fields of strongly convex functions are strongly contracting vector fields, modulo a sign change. A central problem in these fields is the design of efficient fixed point algorithms; recent contributions in this spirit are [18], [13].

Of special interest in this paper are contracting systems in non-Euclidean spaces, i.e., vector fields whose flow is a contraction mapping with respect to a non-Euclidean norm.

In this context, Aminzare and Sontag were the first to highlight the connections between contraction theory and semi-inner products in [1], [2].

This tutorial is based upon the theory of weak pairings recently developed in [6], [9], [10]. We remark that monotone operators over smooth semi-inner product spaces are studied for example in [15]; here we are precisely interested in nonsmooth polyhedral norms, such as the`1and`_∞norms.

For the same reason (lack of differentiability), contraction theory over Finsler manifolds does not directly apply to the non-Euclidean problems of interest here.

Center of Control, Dynamical Systems and Computation, University of California, Santa Barbara, 93106-5070, USA. This work was sup- ported in part by the Defense Threat Reduction Agency under Contract No. HDTRA1-19-1-0017. Figures 1 and 2 are licensed under the CC BY- SA 4.0. (bullo@ucsb.edu, pacisne@gmail.com, {davydov, saber}@ucsb.edu)

This document also briefly reviews some generalizations to Riemannian manifolds. Contraction theory on Riemannian manifolds originates in the influential work by Lohmiller and Slotine [11]. A formal coordinate-free analysis (with connection to monotone operators) is given in [16]. In the differential geometry literature, the study of geodesically monotonic vector fields initiated in [12] and relevant exten- sions were obtained in [5], [17].

This document is intended to be a tutorial and makes the following contributions. First, we provide a unified view of the main theorem on contraction and incremental stability in the context of Euclidean, Riemannian and non-Euclidean spaces. Similarly, we present a unified investigation into fixed point algorithms over these three domains. Second, we consider the setting of strongly contracting vector fields with respect to non-Euclidean norms: we analyze and establish convergence factors for the explicit Euler (from [10]), explicit extragradient, and implicit Euler algorithms. Notably, these results provide a starting point for the generalization of convex analysis and monotone operator theory to the setting of strongly contracting vector fields with respect to the norms

`1 and`_∞. Finally, we include a number of conjectures that will hopefully stimulate further research.

A brief review of matrix measures

We recall the standard`p induced norms,p ∈ {1, 2, ∞}:

kAk2= q

λmax(A^>A), kAk1= max

j∈{1,...,n}

n

X

i=1

|aij|, kAk_∞= max

i∈{1,...,n}

n

X

j=1

|aij|.

where λmax(A^>A) is the largest eigenvalue of A^>A. The matrix measureof A ∈ Rⁿ^×nwith respect to a norm k · k is

µ(A) := lim

h→0⁺

kIn+ hAk − 1

h . (1)

From [7] we recallµ2(A) =¹₂λmax A + A^>, µ1(A) = max

j∈{1,...,n}

ajj+

n

X

i=1,i6=j

|aij|

, µ_∞(A) = µ1(A^>).

For R invertible square, we define kAkp,R = kRAkp and its associated matrix measure µp,R(A) = µp(RAR⁻¹). For P = P^> 0, we write kxk²_P = kxk²_2,P1/2= x^>P x. Matrix measures enjoy numerous properties [7]; we present here only the so-called Lumer’s equalities:

µ_2,P1/2(A) = max

kxk_{2,P 1/2}=1x^>P Ax (2a)

= min{b ∈ R | A^>P + P A 2bP }. (2b) 2021 60th IEEE Conference on Decision and Control (CDC)

December 13-15, 2021. Austin, Texas

(2)

II. CONTRACTION ANDMONOTONEOPERATORS ON THE

EUCLIDEANSPACE(Rⁿ, `2)

We start with a very simple motivating discussion. For b ∈ R, f : R → R is one-sided Lipschitz (osL) if

(x − y)(f (x) − f (y)) ≤ b(x − y)², ∀x, y (3)

⇐⇒ f (x) − f (y) ≤ b(x − y), ∀x > y (4) and iff is continuously differentiable

⇐⇒ f⁰(x) ≤ b, ∀x (5)

We refer to (5) as differential one-sided Lipschitz bound (d- osL). We note that

• f is osL with b = 0 if and only if f weakly decreasing;

• iff is Lipschitz with bound `, then f is osL with b = `, whereas the converse is false;

• finally, for the scalar dynamics ˙x = f (x), the Gr¨onwall lemma implies |x(t) − y(t)| ≤ e^bt|x(0) − y(0)|.

In what follows, we generalize this simple discussion in numerous directions and study its implications.

A. Contraction and Incremental Stability

For a continuously differentiablef : Rⁿ→ Rⁿ, consider

˙x = f (x). (6)

We next state the main theorem of contraction and exponential incremental stability.

Theorem 1 (Equivalences on (Rⁿ, `2)): For P = P^> 0 and c > 0, the following statements are equivalent:

(i) (f (x) − f (y))^>P (x − y) ≤ −ckx − yk²_2,P1/2, for all x, y;

(ii) P Df (x) + Df (x)^>P −2cP for all x, or equivalently µ_2,P1/2(Df (x)) ≤ −c for all x;

(iii) D⁺kx(t) − y(t)k2,P^1/2 ≤ −ckx(t) − y(t)k2,P^1/2, for all solutions x(·), y(·), where D⁺ is the upper right Dini derivative;

(iv) kx(t) − y(t)k_2,P1/2 ≤ e^−ctkx(0) − y(0)k_2,P1/2, for all solutionsx(·), y(·).

A vector field f satisfying any and therefore all of these conditions is said to bec-strongly contacting.

We refer to statement (i) as the one-sided Lipschitz condition (osL) and statement (ii) as the differential one-sided Lipschitz (d-osL) (a.k.a. the Demidovich condition). The last two statements are about differential incremental stability (d- IS) and exponential incremental stability (IS), respectively.

Proof: We include an incomplete sketch of the proof.

Statement (i) implies (ii) by letting y = x + hv for some v ∈ Rⁿ and taking the limit as h → 0⁺. Statement (ii) implies (iii) by Coppel’s inequality [7, Lemma A]. State- ment(iii)implies(iv)by the Gr¨onwall Comparison Lemma.

Statement(iv)implies (i)by a Taylor expansion.

Variations of Theorem 1 hold for (1) forward-invariant convex sets, (2) time-dependent vector fields, and (3) non- differentiable vector fieldsf , where three of the four properties remain equivalent: osL, d-IS, and IS.

For an affinef (x) = Ax + b, the osL condition reads (f (x) − f (y))^>P (x − y) = (x − y)^>A^>P (x − y)

= (x − y)^>A^>P + P A

2 (x − y) ≤ −ckx − yk²_2,P1/2. (7) Lumer’s equalities (2) imply that the smallest number −c ensuring the osL and d-osL conditions is −c = µ_2,P1/2(A).

B. Consequences of Contraction: Equilibria

One of the numerous desirable properties of strongly contracting vector fields is that their flow forgets initial conditions (e.g., see Figure2) and, in the time-invariant case, globally exponentially converges to a unique equilibrium point. These points are illustrated in the next result.

<latexit sha1_base64="N3hfoZuaqNqDBlz3SHiemm3eHYU=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoseCF48V7Qe0oWy2k3bpZhN2N2IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4Zua3H1FpHssHM0nQj+hQ8pAzaqx0/9R3++WKW3XnIKvEy0kFcjT65a/eIGZphNIwQbXuem5i/Iwqw5nAaamXakwoG9Mhdi2VNELtZ/NTp+TMKgMSxsqWNGSu/p7IaKT1JApsZ0TNSC97M/E/r5ua8NrPuExSg5ItFoWpICYms7/JgCtkRkwsoUxxeythI6ooMzadkg3BW355lbQuqt5l1b2rVeq1PI4inMApnIMHV1CHW2hAExgM4Rle4c0Rzovz7nwsWgtOPnMMf+B8/gAHSI2V</latexit>

x0

<latexit sha1_base64="wi0Ie671B5q+pQnuDOsZDzQkN+I=">AAAB6nicdVDLSgNBEOyNrxhfUY9eBoPgKcyEaJJbwIvHiOYByRJmJ7PJkNkHM7PCEvIJXjwo4tUv8ubfOJtEUNGChqKqm+4uL5ZCG4w/nNza+sbmVn67sLO7t39QPDzq6ChRjLdZJCPV86jmUoS8bYSRvBcrTgNP8q43vcr87j1XWkThnUlj7gZ0HApfMGqsdJsO8bBYwmWMMSEEZYTULrEljUa9QuqIZJZFCVZoDYvvg1HEkoCHhkmqdZ/g2Lgzqoxgks8Lg0TzmLIpHfO+pSENuHZni1Pn6MwqI+RHylZo0EL9PjGjgdZp4NnOgJqJ/u1l4l9ePzF+3Z2JME4MD9lykZ9IZCKU/Y1GQnFmZGoJZUrYWxGbUEWZsekUbAhfn6L/SadSJhdlfFMtNaurOPJwAqdwDgRq0IRraEEbGIzhAZ7g2ZHOo/PivC5bc85q5hh+wHn7BFXEjcw=</latexit>

y0

y(t)

<latexit sha1_base64="G60v/zmKfwsWL8x01beEcid3o+c=">AAAB63icdVDLSgMxFM3UV62vqks3wSLUTUmK2HZXdOOygn1AO5RMmmlDk8yQZIQy9BfcuFDErT/kzr8x01ZQ0QMXDufcy733BLHgxiL04eXW1jc2t/LbhZ3dvf2D4uFRx0SJpqxNIxHpXkAME1yxtuVWsF6sGZGBYN1gep353XumDY/UnZ3FzJdkrHjIKbGZNCvb82GxhCoIIYwxzAiuXSJHGo16FdchziyHElihNSy+D0YRTSRTlgpiTB+j2Pop0ZZTweaFQWJYTOiUjFnfUUUkM366uHUOz5wygmGkXSkLF+r3iZRIY2YycJ2S2In57WXiX14/sWHdT7mKE8sUXS4KEwFtBLPH4YhrRq2YOUKo5u5WSCdEE2pdPAUXwten8H/SqVYwquDbi1LzahVHHpyAU1AGGNRAE9yAFmgDCibgATyBZ096j96L97pszXmrmWPwA97bJ9BGjhY=</latexit><latexit sha1_base64="G60v/zmKfwsWL8x01beEcid3o+c=">AAAB63icdVDLSgMxFM3UV62vqks3wSLUTUmK2HZXdOOygn1AO5RMmmlDk8yQZIQy9BfcuFDErT/kzr8x01ZQ0QMXDufcy733BLHgxiL04eXW1jc2t/LbhZ3dvf2D4uFRx0SJpqxNIxHpXkAME1yxtuVWsF6sGZGBYN1gep353XumDY/UnZ3FzJdkrHjIKbGZNCvb82GxhCoIIYwxzAiuXSJHGo16FdchziyHElihNSy+D0YRTSRTlgpiTB+j2Pop0ZZTweaFQWJYTOiUjFnfUUUkM366uHUOz5wygmGkXSkLF+r3iZRIY2YycJ2S2In57WXiX14/sWHdT7mKE8sUXS4KEwFtBLPH4YhrRq2YOUKo5u5WSCdEE2pdPAUXwten8H/SqVYwquDbi1LzahVHHpyAU1AGGNRAE9yAFmgDCibgATyBZ096j96L97pszXmrmWPwA97bJ9BGjhY=</latexit><latexit sha1_base64="G60v/zmKfwsWL8x01beEcid3o+c=">AAAB63icdVDLSgMxFM3UV62vqks3wSLUTUmK2HZXdOOygn1AO5RMmmlDk8yQZIQy9BfcuFDErT/kzr8x01ZQ0QMXDufcy733BLHgxiL04eXW1jc2t/LbhZ3dvf2D4uFRx0SJpqxNIxHpXkAME1yxtuVWsF6sGZGBYN1gep353XumDY/UnZ3FzJdkrHjIKbGZNCvb82GxhCoIIYwxzAiuXSJHGo16FdchziyHElihNSy+D0YRTSRTlgpiTB+j2Pop0ZZTweaFQWJYTOiUjFnfUUUkM366uHUOz5wygmGkXSkLF+r3iZRIY2YycJ2S2In57WXiX14/sWHdT7mKE8sUXS4KEwFtBLPH4YhrRq2YOUKo5u5WSCdEE2pdPAUXwten8H/SqVYwquDbi1LzahVHHpyAU1AGGNRAE9yAFmgDCibgATyBZ096j96L97pszXmrmWPwA97bJ9BGjhY=</latexit><latexit sha1_base64="G60v/zmKfwsWL8x01beEcid3o+c=">AAAB63icdVDLSgMxFM3UV62vqks3wSLUTUmK2HZXdOOygn1AO5RMmmlDk8yQZIQy9BfcuFDErT/kzr8x01ZQ0QMXDufcy733BLHgxiL04eXW1jc2t/LbhZ3dvf2D4uFRx0SJpqxNIxHpXkAME1yxtuVWsF6sGZGBYN1gep353XumDY/UnZ3FzJdkrHjIKbGZNCvb82GxhCoIIYwxzAiuXSJHGo16FdchziyHElihNSy+D0YRTSRTlgpiTB+j2Pop0ZZTweaFQWJYTOiUjFnfUUUkM366uHUOz5wygmGkXSkLF+r3iZRIY2YycJ2S2In57WXiX14/sWHdT7mKE8sUXS4KEwFtBLPH4YhrRq2YOUKo5u5WSCdEE2pdPAUXwten8H/SqVYwquDbi1LzahVHHpyAU1AGGNRAE9yAFmgDCibgATyBZ096j96L97pszXmrmWPwA97bJ9BGjhY=</latexit>

x(t)

<latexit sha1_base64="DO6xEqo2v2AzMk4JoJD7HdETlXk=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BItQLyURQY9FLx4r2A9oQ9lsN+3S3U3YnYil9C948aCIV/+QN/+NmzYHbX0w8Hhvhpl5YSK4Qc/7dgpr6xubW8Xt0s7u3v5B+fCoZeJUU9aksYh1JySGCa5YEzkK1kk0IzIUrB2ObzO//ci04bF6wEnCAkmGikecEsykpyqe98sVr+bN4a4SPycVyNHol796g5imkimkghjT9b0EgynRyKlgs1IvNSwhdEyGrGupIpKZYDq/deaeWWXgRrG2pdCdq78npkQaM5Gh7ZQER2bZy8T/vG6K0XUw5SpJkSm6WBSlwsXYzR53B1wzimJiCaGa21tdOiKaULTxlGwI/vLLq6R1UfO9mn9/Wanf5HEU4QROoQo+XEEd7qABTaAwgmd4hTdHOi/Ou/OxaC04+cwx/IHz+QOByY3f</latexit><latexit sha1_base64="DO6xEqo2v2AzMk4JoJD7HdETlXk=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BItQLyURQY9FLx4r2A9oQ9lsN+3S3U3YnYil9C948aCIV/+QN/+NmzYHbX0w8Hhvhpl5YSK4Qc/7dgpr6xubW8Xt0s7u3v5B+fCoZeJUU9aksYh1JySGCa5YEzkK1kk0IzIUrB2ObzO//ci04bF6wEnCAkmGikecEsykpyqe98sVr+bN4a4SPycVyNHol796g5imkimkghjT9b0EgynRyKlgs1IvNSwhdEyGrGupIpKZYDq/deaeWWXgRrG2pdCdq78npkQaM5Gh7ZQER2bZy8T/vG6K0XUw5SpJkSm6WBSlwsXYzR53B1wzimJiCaGa21tdOiKaULTxlGwI/vLLq6R1UfO9mn9/Wanf5HEU4QROoQo+XEEd7qABTaAwgmd4hTdHOi/Ou/OxaC04+cwx/IHz+QOByY3f</latexit><latexit sha1_base64="DO6xEqo2v2AzMk4JoJD7HdETlXk=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BItQLyURQY9FLx4r2A9oQ9lsN+3S3U3YnYil9C948aCIV/+QN/+NmzYHbX0w8Hhvhpl5YSK4Qc/7dgpr6xubW8Xt0s7u3v5B+fCoZeJUU9aksYh1JySGCa5YEzkK1kk0IzIUrB2ObzO//ci04bF6wEnCAkmGikecEsykpyqe98sVr+bN4a4SPycVyNHol796g5imkimkghjT9b0EgynRyKlgs1IvNSwhdEyGrGupIpKZYDq/deaeWWXgRrG2pdCdq78npkQaM5Gh7ZQER2bZy8T/vG6K0XUw5SpJkSm6WBSlwsXYzR53B1wzimJiCaGa21tdOiKaULTxlGwI/vLLq6R1UfO9mn9/Wanf5HEU4QROoQo+XEEd7qABTaAwgmd4hTdHOi/Ou/OxaC04+cwx/IHz+QOByY3f</latexit><latexit sha1_base64="DO6xEqo2v2AzMk4JoJD7HdETlXk=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BItQLyURQY9FLx4r2A9oQ9lsN+3S3U3YnYil9C948aCIV/+QN/+NmzYHbX0w8Hhvhpl5YSK4Qc/7dgpr6xubW8Xt0s7u3v5B+fCoZeJUU9aksYh1JySGCa5YEzkK1kk0IzIUrB2ObzO//ci04bF6wEnCAkmGikecEsykpyqe98sVr+bN4a4SPycVyNHol796g5imkimkghjT9b0EgynRyKlgs1IvNSwhdEyGrGupIpKZYDq/deaeWWXgRrG2pdCdq78npkQaM5Gh7ZQER2bZy8T/vG6K0XUw5SpJkSm6WBSlwsXYzR53B1wzimJiCaGa21tdOiKaULTxlGwI/vLLq6R1UfO9mn9/Wanf5HEU4QROoQo+XEEd7qABTaAwgmd4hTdHOi/Ou/OxaC04+cwx/IHz+QOByY3f</latexit>

circle with radius

<latexit sha1_base64="FjWk5HUIbx2Zx22MHiL80m6vxXM=">AAAB/XicbVC5TsNAEB1zhnCFo6NZESFRRXYaKCNoKINEDimxovV6nayyPrQ7BgUr4ldoKECIlv+g429YJy4g4UkjPb03szvzvEQKjbb9ba2srq1vbJa2yts7u3v7lYPDto5TxXiLxTJWXY9qLkXEWyhQ8m6iOA09yTve+Dr3O/dcaRFHdzhJuBvSYSQCwSgaaVA5ZkIxycmDwBFR1BepJuVBpWrX7BnIMnEKUoUCzUHlq+/HLA15hExSrXuOnaCbUYXCPD4t91PNE8rGdMh7hkY05NrNZttPyZlRfBLEylSEZKb+nshoqPUk9ExnSHGkF71c/M/rpRhcupmIkhR5xOYfBakkGJM8CuILxRnKiSGUKWF2JWxEFWVoAstDcBZPXibtes2xa85tvdq4KuIowQmcwjk4cAENuIEmtIDBIzzDK7xZT9aL9W59zFtXrGLmCP7A+vwBfDyUkQ==</latexit><latexit sha1_base64="FjWk5HUIbx2Zx22MHiL80m6vxXM=">AAAB/XicbVC5TsNAEB1zhnCFo6NZESFRRXYaKCNoKINEDimxovV6nayyPrQ7BgUr4ldoKECIlv+g429YJy4g4UkjPb03szvzvEQKjbb9ba2srq1vbJa2yts7u3v7lYPDto5TxXiLxTJWXY9qLkXEWyhQ8m6iOA09yTve+Dr3O/dcaRFHdzhJuBvSYSQCwSgaaVA5ZkIxycmDwBFR1BepJuVBpWrX7BnIMnEKUoUCzUHlq+/HLA15hExSrXuOnaCbUYXCPD4t91PNE8rGdMh7hkY05NrNZttPyZlRfBLEylSEZKb+nshoqPUk9ExnSHGkF71c/M/rpRhcupmIkhR5xOYfBakkGJM8CuILxRnKiSGUKWF2JWxEFWVoAstDcBZPXibtes2xa85tvdq4KuIowQmcwjk4cAENuIEmtIDBIzzDK7xZT9aL9W59zFtXrGLmCP7A+vwBfDyUkQ==</latexit><latexit sha1_base64="FjWk5HUIbx2Zx22MHiL80m6vxXM=">AAAB/XicbVC5TsNAEB1zhnCFo6NZESFRRXYaKCNoKINEDimxovV6nayyPrQ7BgUr4ldoKECIlv+g429YJy4g4UkjPb03szvzvEQKjbb9ba2srq1vbJa2yts7u3v7lYPDto5TxXiLxTJWXY9qLkXEWyhQ8m6iOA09yTve+Dr3O/dcaRFHdzhJuBvSYSQCwSgaaVA5ZkIxycmDwBFR1BepJuVBpWrX7BnIMnEKUoUCzUHlq+/HLA15hExSrXuOnaCbUYXCPD4t91PNE8rGdMh7hkY05NrNZttPyZlRfBLEylSEZKb+nshoqPUk9ExnSHGkF71c/M/rpRhcupmIkhR5xOYfBakkGJM8CuILxRnKiSGUKWF2JWxEFWVoAstDcBZPXibtes2xa85tvdq4KuIowQmcwjk4cAENuIEmtIDBIzzDK7xZT9aL9W59zFtXrGLmCP7A+vwBfDyUkQ==</latexit><latexit sha1_base64="FjWk5HUIbx2Zx22MHiL80m6vxXM=">AAAB/XicbVC5TsNAEB1zhnCFo6NZESFRRXYaKCNoKINEDimxovV6nayyPrQ7BgUr4ldoKECIlv+g429YJy4g4UkjPb03szvzvEQKjbb9ba2srq1vbJa2yts7u3v7lYPDto5TxXiLxTJWXY9qLkXEWyhQ8m6iOA09yTve+Dr3O/dcaRFHdzhJuBvSYSQCwSgaaVA5ZkIxycmDwBFR1BepJuVBpWrX7BnIMnEKUoUCzUHlq+/HLA15hExSrXuOnaCbUYXCPD4t91PNE8rGdMh7hkY05NrNZttPyZlRfBLEylSEZKb+nshoqPUk9ExnSHGkF71c/M/rpRhcupmIkhR5xOYfBakkGJM8CuILxRnKiSGUKWF2JWxEFWVoAstDcBZPXibtes2xa85tvdq4KuIowQmcwjk4cAENuIEmtIDBIzzDK7xZT9aL9W59zFtXrGLmCP7A+vwBfDyUkQ==</latexit> e ^ct<latexit sha1_base64="umOeAwXv+gFhtCh4wkqBlaWBa8g=">AAAB7nicbVBNS8NAEJ34WetX1aOXYBG8WBIR9Fj04rGC/YC2ls120i7dbMLuRCihP8KLB0W8+nu8+W/ctjlo64OBx3szzMwLEikMed63s7K6tr6xWdgqbu/s7u2XDg4bJk41xzqPZaxbATMohcI6CZLYSjSyKJDYDEa3U7/5hNqIWD3QOMFuxAZKhIIzslITH7NzTpNeqexVvBncZeLnpAw5ar3SV6cf8zRCRVwyY9q+l1A3Y5oElzgpdlKDCeMjNsC2pYpFaLrZ7NyJe2qVvhvG2pYid6b+nshYZMw4CmxnxGhoFr2p+J/XTim87mZCJSmh4vNFYSpdit3p725faOQkx5YwroW91eVDphknm1DRhuAvvrxMGhcV36v495fl6k0eRwGO4QTOwIcrqMId1KAOHEbwDK/w5iTOi/PufMxbV5x85gj+wPn8AT93j38=</latexit><latexit sha1_base64="umOeAwXv+gFhtCh4wkqBlaWBa8g=">AAAB7nicbVBNS8NAEJ34WetX1aOXYBG8WBIR9Fj04rGC/YC2ls120i7dbMLuRCihP8KLB0W8+nu8+W/ctjlo64OBx3szzMwLEikMed63s7K6tr6xWdgqbu/s7u2XDg4bJk41xzqPZaxbATMohcI6CZLYSjSyKJDYDEa3U7/5hNqIWD3QOMFuxAZKhIIzslITH7NzTpNeqexVvBncZeLnpAw5ar3SV6cf8zRCRVwyY9q+l1A3Y5oElzgpdlKDCeMjNsC2pYpFaLrZ7NyJe2qVvhvG2pYid6b+nshYZMw4CmxnxGhoFr2p+J/XTim87mZCJSmh4vNFYSpdit3p725faOQkx5YwroW91eVDphknm1DRhuAvvrxMGhcV36v495fl6k0eRwGO4QTOwIcrqMId1KAOHEbwDK/w5iTOi/PufMxbV5x85gj+wPn8AT93j38=</latexit><latexit sha1_base64="umOeAwXv+gFhtCh4wkqBlaWBa8g=">AAAB7nicbVBNS8NAEJ34WetX1aOXYBG8WBIR9Fj04rGC/YC2ls120i7dbMLuRCihP8KLB0W8+nu8+W/ctjlo64OBx3szzMwLEikMed63s7K6tr6xWdgqbu/s7u2XDg4bJk41xzqPZaxbATMohcI6CZLYSjSyKJDYDEa3U7/5hNqIWD3QOMFuxAZKhIIzslITH7NzTpNeqexVvBncZeLnpAw5ar3SV6cf8zRCRVwyY9q+l1A3Y5oElzgpdlKDCeMjNsC2pYpFaLrZ7NyJe2qVvhvG2pYid6b+nshYZMw4CmxnxGhoFr2p+J/XTim87mZCJSmh4vNFYSpdit3p725faOQkx5YwroW91eVDphknm1DRhuAvvrxMGhcV36v495fl6k0eRwGO4QTOwIcrqMId1KAOHEbwDK/w5iTOi/PufMxbV5x85gj+wPn8AT93j38=</latexit><latexit sha1_base64="umOeAwXv+gFhtCh4wkqBlaWBa8g=">AAAB7nicbVBNS8NAEJ34WetX1aOXYBG8WBIR9Fj04rGC/YC2ls120i7dbMLuRCihP8KLB0W8+nu8+W/ctjlo64OBx3szzMwLEikMed63s7K6tr6xWdgqbu/s7u2XDg4bJk41xzqPZaxbATMohcI6CZLYSjSyKJDYDEa3U7/5hNqIWD3QOMFuxAZKhIIzslITH7NzTpNeqexVvBncZeLnpAw5ar3SV6cf8zRCRVwyY9q+l1A3Y5oElzgpdlKDCeMjNsC2pYpFaLrZ7NyJe2qVvhvG2pYid6b+nshYZMw4CmxnxGhoFr2p+J/XTim87mZCJSmh4vNFYSpdit3p725faOQkx5YwroW91eVDphknm1DRhuAvvrxMGhcV36v495fl6k0eRwGO4QTOwIcrqMId1KAOHEbwDK/w5iTOi/PufMxbV5x85gj+wPn8AT93j38=</latexit>

Fig. 1: Exponential incremental stability of contracting vector fields.

The distance between two trajectories decreases exponentially fast.

Theorem 2 (Equilibria of contracting vector fields): For a time-invariant vector fieldf that is c-strongly contracting with respect to k · k_2,P1/2,P = P^> 0,

(i) the flow off is a contraction, i.e., the distance between solutions exponentially decreases with ratec, and (ii) there exists a unique equilibrium x^∗ that is globally

exponentially stable with global Lyapunov functions x 7→ kx − x^∗k²_2,P1/2 and x 7→ kf (x)k²_2,P1/2. Proof: We include an incomplete sketch of the proof.

Theorem 5(iv) immediately implies (i) and that, for any positive τ , the flow map of the vector field at time τ is a contraction map with constant e^−cτ. The fixed point of this contraction map is either a period orbit with periodτ (which is impossible) or a fixed point of the flow map. The global Lyapunov functions follow from direct computation.

C. Equilibrium Computation via Forward Step Method The study of monotone operators is closely related to the study of contracting vector fields. As it is classic in the study of monotone operators, we here aim to provide an algorithm to compute the equilibrium points of a vector field f (equivalently regarded as an operator):

x^∗∈ zero(f ) ⇐⇒ x^∗∈ fixed(Id +αf ), (8) for anyα > 0, where Id is the identity map. Here we define zero(f ) = {x ∈ Rⁿ| f (x) = 0} and fixed(Id +αf ) = {x ∈ Rⁿ | x = (Id +αf )x}. A map f is (globally) `-Lipschitz continuousif

kf (x) − f (y)k_2,P1/2≤ `kx − yk_2,P1/2. (9)

(3)

for all x, y. We define the operator condition number of a c-strongly contracting and `-Lipschitz continuous map f by

κ = `/c ≥ 1. (10)

Remark 3 (Literature comparison): In the literature on monotone operators, givenP = P^> 0, the map g : Rⁿ→ Rⁿ isc-strongly monotone if

(g(x) − g(y))^>P (x − y) ≥ ckx − yk²_2,P1/2. (11) Clearly, g is c-strongly monotone if and only if −g is c- strongly contracting.

Next, we compare the operator condition number of a contracting affine f (x) = Ax + b, A ∈ Rⁿ^×n, with the standard contraction number of A. First, recall that, given a norm k·k, the condition number of a square invertible matrix A is κ(A) = kAkkA⁻¹k. Second, for the P^1/2-weighted`2

norm, we know from (7) that the contraction rate off equals µ_2,P1/2(A). Accordingly, given a norm k · k, the operator condition number of a square matrixA with µ(A) < 0 is

κµ(A) = kAk

|µ(A)|. (12)

From [7], note that µ(A) < 0 implies kA⁻¹k ≤ 1/|µ(A)|.

Therefore, κ(A) ≤ κµ(A). One can show that the two condition numbers coincide forA = A^> andP = In. Given a start pointx0∈ Rⁿ, the forward step method for the operator f , i.e., the explicit Euler integration algorithm for the vector fieldf , is:

xk+1= (Id +αf )xk = xk+ αf (xk). (13) Theorem 4: (Optimal step size and contraction factor of forward step method) For P = P^> 0, consider a map f : Rⁿ → Rⁿ with strong contraction rate c > 0, Lipschitz constant` > 0, and condition number κ = `/c. Then

(i) the map Id +αf is a contraction map with respect to k · k_2,P1/2 for

0 < α < 2 cκ²,

(ii) the step size minimizing the contraction factor and the minimum contraction factor (that is, the minimal Lipschitz constant ofId +αf ) are

α^∗_E= 1 cκ²,

`^∗E= 1 − 1

κ²

^1/2

= 1 − 1

2κ² + O 1 κ⁴

. (14)

Proof: We only sketch the standard proof here:

k(Id +αf )x − (Id +αf )yk²_2,P1/2

= kx − y + α(f (x) − f (y))k²_2,P1/2

= kx − yk²_2,P1/2+ 2α(f (x) − f (y))^>P (x − y) + α²kf (x) − f (y)k²_2,P1/2

≤ (1 − 2αc + α²`²)kx − yk²_2,P1/2.

It is easy to check that (1 − 2αc + α²`²) < 1 if and only if0 < α < 2c/`² and that the minimal contraction factor is (1 − c²/`²)^1/2 atα^∗= c/`².

III. CONTRACTIONTHEORY ANDMONOTONE

OPERATORS ONRIEMANNIANMANIFOLDS

In this section we consider a Riemannian manifold(M, G) with associated Levi-Civita connection ∇, geodesic distance d_G, and parallel transportP (γ) along a geodesic arc γ. Let hh·, ·ii_G denote the inner product associated to G and γ⁰ denote the velocity vector along a geodesic arc.

Loosely speaking, a vector field X on a Riemannian manifold is geodesically contracting (−X is geodesically monotone) if the first variation of the length of each geodesic arcγ, with infinitesimal variation equal to the restriction of X to γ, is nonpositive.

<latexit sha1_base64="Z7aKgWufqTjB/Ih4RC7ftI5GSqw=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69BIvgqSRS1GPBixehQr+gDWWz3bRLN5u4Oyktob/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM82PBNTrOt5Xb2Nza3snvFvb2Dw6PiscnTR0lirIGjUSk2j7RTHDJGshRsHasGAl9wVr+6G7ut8ZMaR7JOk5j5oVkIHnAKUEjefXepItsgjpIH2a9YskpOwvY68TNSAky1HrFr24/oknIJFJBtO64ToxeShRyKtis0E00iwkdkQHrGCpJyLSXLo6e2RdG6dtBpExJtBfq74mUhFpPQ990hgSHetWbi/95nQSDWy/lMk6QSbpcFCTCxsieJ2D3uWIUxdQQQhU3t9p0SBShaHIqmBDc1ZfXSfOq7F6XncdKqVrJ4sjDGZzDJbhwA1W4hxo0gMITPMMrvFlj68V6tz6WrTkrmzmFP7A+fwA95JJd</latexit>

TxM

<latexit sha1_base64="6xjst/J04xEQ5w8Npz/9m3suXoM=">AAAB9HicbVBNS8NAEN3Ur1q/qh69BIvgqSQi6rHgxYtQoV/QhrLZTtqlm03cnRRD6O/w4kERr/4Yb/4bt20O2vpg4PHeDDPz/FhwjY7zbRXW1jc2t4rbpZ3dvf2D8uFRS0eJYtBkkYhUx6caBJfQRI4COrECGvoC2v74dua3J6A0j2QD0xi8kA4lDzijaCSv0U97CE+og+x+2i9XnKozh71K3JxUSI56v/zVG0QsCUEiE1TrruvE6GVUIWcCpqVeoiGmbEyH0DVU0hC0l82PntpnRhnYQaRMSbTn6u+JjIZap6FvOkOKI73szcT/vG6CwY2XcRknCJItFgWJsDGyZwnYA66AoUgNoUxxc6vNRlRRhiankgnBXX55lbQuqu5V1Xm4rNQu8ziK5IScknPikmtSI3ekTpqEkUfyTF7JmzWxXqx362PRWrDymWPyB9bnDz9ykl4=</latexit>

TyM

<latexit sha1_base64="IhNGiu7P9R2OeElFBZASufY9lL8=">AAACoHicdZFLTxsxEMed7YtuHzx67MUiqtRDFdkIQbhR0QMXVFAJoGajaOydTSzs3ZXtRUSrVT9Fr+Vz8W3whqCWKoxk6ef5ex6eEaVWzjN224mePX/x8tXK6/jN23fvV9fWN85cUVmJA1nowl4IcKhVjgOvvMaL0iIYofFcXB60+vkVWqeK/NTPShwZmOQqUxJ8cA2TCRgD4/p61ozXuqzHGOOc0xb47g4LsLfX3+J9ylspWJcs7Hi83vmVpIWsDOZeanBuyFnpRzVYr6TGJk4qhyXIS5jgMGAOBt0XdzWZw6i+nnff0E9BS2lW2HByT+fef2NrMM7NjAgvDfip+19rncu0YeWz/qhWeVl5zOV9oazS1Be0HQVNlUXp9SwASKtC11ROwYL0YWCPqgjTxHHyDcM3LR6FQl91OQWBvk7mZTPlm/qUN3XishQzqHS4Jyb9y04/cJz8QP9UDlHodEkmkS3NFFb2sBf6NJxt9fhOj51sd/e3F8tbIR/JJvlMONkl++SQHJMBkaQgv8kfchNtRofR9+jk/mnUWcR8II8s+nkHKj/Vhg==</latexit>

xy

<latexit sha1_base64="mQgQjaQWfh1aZop/hthxUHJJeq4=">AAACmXicdVFda9swFFXcfbTeV7s+9kUsDDoYwXZN0r21bA+lMOho0wbiUCT5OhGVbCNdlwZj+g/22v21/ZvJacrWkV4QHJ2je8/VvbxU0mIQ/O54a8+ev3i5vuG/ev3m7bvNrffntqiMgKEoVGFGnFlQMochSlQwKg0wzRVc8KuvrX5xDcbKIj/DeQkTzaa5zKRg2FKj3ZtPl5vdoPdlvx/FfRr0gmAQRmELokG8F9PQMW10yTJOLrc6t0laiEpDjkIxa8dhUOKkZgalUND4SWWhZOKKTWHsYM402M/2eroAk/pm0XZDPzotpVlh3MmRLth/c2umrZ1r7l5qhjP7v9aSq7Rxhdn+pJZ5WSHk4t4oqxTFgrYzoKk0IFDNHWDCSNc1FTNmmEA3qUcuXDe+n3wD900D353RoSpnjAPWycI2k9jUZ2FTJzZLIWOVcvdEp3+xVQ/YT04Bn6rBC5WuqMSzlZXcyh72Qp8G51Ev7PeiH3H3IF4ub53skA9kl4RkQA7IETkhQyLIjPwkd+SXt+Mdekfe8f1Tr7PM2SaPwjv9A23s0cg=</latexit>

X(x)

<latexit sha1_base64="COECOoRCmn3lhNZRsmC/TMtX9Ac=">AAACmXicdVFda9swFFW8buu8r3Z77ItoKHQwgu2ZpHtr2R7KYNDSpg3EoUjydSIq2Ua6LgvG7B/sdftr/TeV05S1Jb0gODpH956re3mppMUguO54z9aev3i5/sp//ebtu/cbmx/ObFEZAUNRqMKMOLOgZA5DlKhgVBpgmis455ffWv38CoyVRX6K8xImmk1zmUnBsKVGu/NPFxvdoPd1rx/FfRr0gmAQRmELokH8JaahY9rokmUcXWx2fidpISoNOQrFrB2HQYmTmhmUQkHjJ5WFkolLNoWxgznTYD/bq+kCTOpfi7YbuuO0lGaFcSdHumDv59ZMWzvX3L3UDGf2sdaSq7RxhdnepJZ5WSHk4tYoqxTFgrYzoKk0IFDNHWDCSNc1FTNmmEA3qQcuXDe+n3wH900DP53RgSpnjAPWycI2k9jUp2FTJzZLIWOVcvdEp/+xVXfYT04An6rBC5WuqMSzlZXcyu72Qp8GZ1Ev7Pei47i7Hy+Xt062yDbZJSEZkH1ySI7IkAgyI3/IX/LP2/IOvEPvx+1Tr7PM+UgehHdyA3Af0ck=</latexit>

X(y)

<latexit sha1_base64="YMH1mR4d9FusYDCqvoIkVRONryE=">AAACpnicdZHNbhMxEMed5assH03hyMUiQgQJRXZVtemtCA5cQEU0TaVsFGzvbGLV3l3Zs1Wj1YoH4QoPxdvgTVNBUTqSpZ/n75nxzMjSaI+M/e5Ed+7eu/9g62H86PGTp9vdnWenvqicgpEqTOHOpPBgdA4j1GjgrHQgrDQwlufvW318Ac7rIj/BZQlTK+a5zrQSGFyz7nYyF9aKWX25bF732ZtZt8cGjDHOOW2BH+yzAIeHw10+pLyVgvXI2o5nO53vSVqoykKOygjvJ5yVOK2FQ60MNHFSeSiFOhdzmATMhQX/1l/MVzCtL1c9NPRV0FKaFS6cHOnK+29sLaz3SyvDSytw4f/XWucmbVJhNpzWOi8rhFxdFcoqQ7Gg7UBoqh0oNMsAQjkdfk3VQjihMIztRhVpmzhOPkBo08GnUOidKRdCAtbJqmymsalPeFMnPkshE5UJ98Smf9mba46Tr4C35ZCFSTdkktnGTGFl13uht8Pp7oDvD9iXvd4RWy9vi7wgL0mfcHJAjshHckxGRJGK/CA/ya+oH32ORtH46mnUWcc8Jzcs+vYH0PXWgw==</latexit>

0xy(0) <latexit sha1_base64="VrV7tJAgCfBvU4BcXWkDYeV7Yto=">AAACpnicdZHNbhMxEMed5assH03hyMUiQgQJRXZVtemtCA5cQEU0TaVsFGzvbGLV3l3Zs1Wj1YoH4QoPxdvgTVNBUTqSpZ/n75nxzMjSaI+M/e5Ed+7eu/9g62H86PGTp9vdnWenvqicgpEqTOHOpPBgdA4j1GjgrHQgrDQwlufvW318Ac7rIj/BZQlTK+a5zrQSGFyz7nYyF9aKWX25bF73+ZtZt8cGjDHOOW2BH+yzAIeHw10+pLyVgvXI2o5nO53vSVqoykKOygjvJ5yVOK2FQ60MNHFSeSiFOhdzmATMhQX/1l/MVzCtL1c9NPRV0FKaFS6cHOnK+29sLaz3SyvDSytw4f/XWucmbVJhNpzWOi8rhFxdFcoqQ7Gg7UBoqh0oNMsAQjkdfk3VQjihMIztRhVpmzhOPkBo08GnUOidKRdCAtbJqmymsalPeFMnPkshE5UJ98Smf9mba46Tr4C35ZCFSTdkktnGTGFl13uht8Pp7oDvD9iXvd4RWy9vi7wgL0mfcHJAjshHckxGRJGK/CA/ya+oH32ORtH46mnUWcc8Jzcs+vYH0yjWhA==</latexit>

xy0 (1)

<latexit sha1_base64="u22NARKKbTUAidGfE72kboV2+Qc=">AAACqHicdVHbbtNAEN2YWzGXJvDIy4oIqSAU2cZKylsRPPCCFETTFMVRtLseJ6vu2tbuuDSyLL6EV/gm/oZ1mgqK0pFGOnvmcnZmeKmkxSD43fFu3b5z997eff/Bw0eP97u9Jye2qIyAiShUYU45s6BkDhOUqOC0NMA0VzDlZ+/b+PQcjJVFfozrEuaaLXOZScHQUYtuL1kyrdmivjjAl2vnzaLbDwZvD4dRPKTBIAhGYRS2IBrFb2IaOqa1PtnaeNHrfE/SQlQachSKWTsLgxLnNTMohYLGTyoLJRNnbAkzB3Omwb6258sNmNcXmyka+sLFUpoVxnmOdMP+W1szbe1ac5epGa7s/7GW3BWbVZgdzmuZlxVCLi6FskpRLGi7EppKAwLV2gEmjHS/pmLFDBPoFndNhevG95MP4MY08MkJvVPlinHAOtnIZhKb+jhs6sRmKWSsUu6d6PQvtuoK+8kXwJt68EKlOzrxbGcnd7Kru9CbwUk0CIeD6HPcP4q3x9sjz8hzckBCMiJH5CMZkwkR5Bv5QX6SX94rb+xNva+XqV5nW/OUXDOP/wE0IdeL</latexit>

x(t)y(t)

Fig. 2: Contractivity of a vector field X on a Riemannian manifold:

the length of the geodesic curve γxy connecting any two points x and y decreases along the flow of X, as a function of the inner product between X and the geodesic velocity vector at x and y.

A. Contraction and Incremental Stability

We consider a time-independent vector fieldX

˙x = X(x). (15)

Theorem 5 (Equivalences on(M, G)): For a Riemannian manifold (M, G) and c > 0, the following statements are equivalent:

(i) for any x, y ∈ M and geodesic curve γxy : [0, 1] → M withγxy(0) = x, γxy(1) = y,

hhX(y), γ_xy⁰ (1)ii_G− hhX(x), γ_xy⁰ (0)ii_G≤ −c d_G(x, y)²; (ii) for all vx∈ TxM

hhAX(x)vx, vxii_G≤ −ckvxk²_G,

whereAX(x) : TxM →TxM is the covariant differential of X defined by AX(x)vx= ∇vxX(x);

(iii) D⁺d_G(x(t), y(t)) ≤ −c d_G(x(t), y(t)), for all solutions x(·), y(·);

(iv) d_G(x(t), y(t)) ≤ e^−ctd_G(x(0), y(0)), for all solutions x(·), y(·).

A vector field X satisfying any and therefore all of these conditions is said to bec-strongly contacting.

Proof: We refer to the appropriate references. The equivalence between property (i) and property (ii) is given in [12], [5]. The implication(ii)=⇒(iii) and(iv)is studied in [16]. As before, the equivalence between statement (iii) and statement (iv)is independent of the vector fieldX and related to the Gr¨onwall comparison lemma.

(4)

Here are some comments drawing a parallel between Theorems 5 and 1. First, condition (i) is known [12], [5]

to be equivalent to either of the following conditions hhγ⁰_xy(t), X(γ(t)ii_G+ ckγ_xy⁰ (0)k²_Gt is monotone decreasing, hhP (γyx)_y→xX(y) − X(x), γ_yx⁰ (0)ii_G≤ −c d_G(x, y)², where P (γyx)y→x : TyM → TxM is the parallel transport along the geodesic from y to x. It is easy to see that, when (M, G) is the Euclidean space with the standard `2 inner product, condition(i) coincides with the one-side Lipschitz condition in Theorem 1(i).

Second, we clarify that statement (ii) can easily be, and usually is, written in components. For every x ∈ M and in a coordinate chart (x¹, . . . , xⁿ) in a neighborhood of x, statement (ii)is equivalent to the linear matrix inequality:

Gki

∂X^k

∂x^` +∂X^k

∂xⁱ Gk`+∂Gi`

∂x^j X^j

−2c[Gi`], (16) or, in matrix form, letting G denote both the Riemannian metric as well as its matrix coordinate representation,

G(x)∂X

∂x(x) +∂X

∂x(x)^>G(x) + ˙G(x) −2cG(x). (17) This is the classic contraction condition given in [11], that generalizes the classic Demidovich condition in Theo- rem1(ii). The parallel between Theorem5(iii)and(iv)versus Theorem1(iii)and(iv) is evident.

B. Consequences of Contraction: Equilibria

In the interest of brevity we do not replicate Theorem2, whose extension to the Riemannian setting naturally holds.

C. Equilibrium Computation via Forward Step Method We start with two useful definitions. Recall that a Rie- mannian manifold M is complete if, for everyvx∈ TxM, the geodesic curveγvx(t) starting at vxat time0 is defined for all t ≥ 0. Accordingly, the exponential map exp_x: TxM → M is defined by exp_x(vx) = γvx(1). A vector field X is `- Lipschitz continuousif

kP (γxy)_x→yX(x) − X(y)k_G≤ ` d_G(x, y), (18) for any x, y ∈ M. Here we assume for simplicity that the geodesic γxy from x to y is unique.

Given a start pointx0∈ M, the forward step method for the operator f , i.e., the explicit Euler integration algorithm for the vector fieldf , is:

xk+1= exp_x_k(αX(xk)). (19) The following result is given in [17, Theorem 5.1].

Theorem 6: (Riemannian forward step method)Consider a vector field X on a Riemannian manifold (M, G) with strong contraction rate c > 0 and Lipschitz constant ` > 0.

Then, for 0 < α < 2c/`², sequence {xk} converges to the unique equilibrium point of X.

To the best of the authors’ knowledge, it is an open conjecture whether the algorithm x 7→ exp_x_k(αX(xk)) given in equation (19) is a Banach contraction mapping.

Practical implementations of the Riemannian forward step algorithms may rely upon an appropriate retraction (as an easily computable replacement of the exponential map).

IV. CONTRACTIONTHEORY ANDMONOTONE

OPERATORS ONNON-EUCLIDEANSPACES

We now consider non-Euclidean spaces including, for example, Rⁿ equipped with either the `1 or `_∞ norms.

A. Linear Algebra Detour: Weak Pairings

We briefly review the notion and the properties of a weak pairing on Rⁿ from [6]. A weak pairing (WP) on Rⁿ is a mapJ·, ·K : R

n× Rⁿ→ R satisfying:

(i) (Sub-additivity and continuity of first argument) Jx1+ x2, yK ≤ Jx1, yK + Jx2, yK, for all x1, x2, y ∈ Rⁿ andJ·, ·K is continuous in its first argument,

(ii) (Weak homogeneity)Jαx, yK = Jx, αyK = α Jx, yK and J−x, −yK = Jx, yK, for all x, y ∈ R

n, α ≥ 0, (iii) (Positive definiteness)Jx, xK > 0, for all x 6= 0n, (iv) (Cauchy-Schwarz inequality)

|Jx, yK | ≤ Jx, xK

1/2

Jy, yK

1/2, for allx, y ∈ Rⁿ. For every norm k · k on Rⁿ, there exists a (possibly not unique) WPJ·, ·K such that kxk

2=Jx, xK, for every x ∈ R

n. When k · k is the`2 norm, the WP coincides with the usual inner product. A WPJ·, ·K satisfies Deimling’s inequality if

Jx, yK ≤ kyk lim

h→0⁺

ky + hxk − kyk

h ,

for everyx, y ∈ Rⁿ. A WP satisfying Deimling’s inequality also satisfies, for allA ∈ R^n×n, the Lumer’s equality

µ(A) = sup

x6=0n

JAx, xK

kxk² . (20)

For invertibleR ∈ Rⁿ^×n, we define the weighted sign WP J·, ·K1,Rand the weighted max WP J·, ·K_∞,R by

Jx, yK1,R= kRyk1sign(Ry)^>Rx, (21) Jx, yK_∞,R = max

i∈I∞(Ry)(Rx)i(Ry)i, (22) whereI_∞(x) = {i ∈ {1, . . . , n} | xi = kxk_∞}. It can be shown that, forp ∈ {1, ∞} and invertible matrix R ∈ Rⁿ^×n, we have kRxk²_p=Jx, xKp,RandJ·, ·Kp,Rsatisfies Deimling’s inequality. We refer to [6] for a detailed discussion on WPs and formulas for arbitraryp ∈ [1, ∞].

B. Contraction and Incremental Stability

For a continuously differentiablef : Rⁿ → Rⁿ, consider

˙x = f (x). (23)

Theorem 7 (Equivalences on(Rⁿ, k · k)): For a norm k·k with matrix measureµ(·) and compatible WPJ·, ·K satisfying Deimling’s inequality, and c > 0, the following statements are equivalent:

(i) Jf (x) − f (y), x − yK ≤ −ckx − yk

2 for allx, y, (ii) JDf (x)v, vK ≤ −ckvk

2, for allv, x, or µ(Df (x)) ≤ −c, for all x,

(iii) D⁺kx(t) − y(t)k ≤ −ckx(t) − y(t)k, for all solutions x(·), y(·),

(iv) kx(t) − y(t)k ≤ e^−c(t)kx(0) − y(0)k, for all solutions x(·), y(·) .

Proof: We refer the reader to [6].

(5)

Norm WP Matrix measure and Lumer equality kxk1=X

i

|xi|

Jx, yK1= kyk1sign(y)^>x

µ1(A) = max

j∈{1,...,n}

ajj +X

i6=j|aij|

= sup

kxk1=1

sign(x)^>Ax

kxk_∞= max

i |xi| Jx, yK_∞= max

i∈I∞(y)yixi

µ_∞(A) = max

i∈{1,...,n}

aii+X

j6=i|aij|

= max

kxk∞=1 max

i∈I∞(x)xi(Ax)i

TABLE I: Table of norms, WPs, and matrix measures for `1, and `∞norms. We define I∞(x) = {i ∈ {1, . . . , n} | |xi| = kxk∞}.

C. Consequences of Contraction: Equilibria

In the interest of brevity we do not replicate Theorem2, whose extension to non-Euclidean setting naturally holds.

D. Equilibrium Computation via Forward Step Method Consider the continuously differentiable dynamics ˙x = f (x). Let k · k denote a norm with compatible WP J·, ·K.

Assume the vector fieldf is c-strongly contracting, i.e., Jf (x) − f (y), x − yK ≤ −ckx − yk

2, (24)

and (globally) Lipschitz continuous with constant`, i.e., kf (x) − f (y)k ≤ `kx − yk, (25) for anyx, y. Next we summarize Theorem 1 from [10].

Theorem 8 (Forward step method on WP spaces):

Consider a norm k · k with compatible WP J·, ·K. Let the continuously differentiable function f be c-strongly contracting, have Lipschitz constant `, and have condition numberκ = `/c ≥ 1. Then

(i) the map Id +αf is a contraction map with respect to k · k for

0 < α < 1 cκ(1 + κ),

(ii) the step size minimizing the contraction factor and the minimum contraction factor are

αnE^∗ = 1 c

1 2κ²− 3

8κ³ + O 1 κ⁴

, (26)

`^∗_nE= 1 − 1 4κ² + 1

8κ³ + O 1 κ⁴

. (27) Compared to the forward step method for contracting systems in the Euclidean space in Theorem 4, the optimal step size is smaller (by a factor of 2 and by higher order terms) and the optimal contraction factor is larger (the gap is larger by a factor of 2 and by higher order terms).

Example 9: Consider the affine system ˙x = Ax+b, where A =−10 2.5

9 −3

andb =−19 20

. We compute

µ2(A) = λmax(¹₂(A + A^>)) = λmax−10 5.75 5.75 −3

= 0.231.

Therefore, this system is not contracting with respect to

`2 norm and Theorem 4 is not applicable for finding its equilibrium point. However,

µ1(A) = −0.5 < 0.

Moreover, we have kA(x − y)k1 ≤ kAk1kx − yk1. Thus, with respect to the `1 norm, the affine system is strongly contracting with rate 0.5 and Lipschitz continuous with Lipschitz constant kAk1. Now we can use Theorem 8 for the`1norm and show thatI2+ α(Ax + b) is contracting for every0 < α < |µ1(A)|

kAk1(|µ1(A)| + kAk1). It is an open conjecture whether a version of Theorem 8 holds for nonsmooth vector fields. We refer to [10] for additional results on the optimal step size and acceleration results for the norms`1 and`_∞.

E. Comments on Implicit Algorithms

We here review the implicit Euler integration scheme and show its basic properties for strongly contracting vector fields; the original reference for this material is [7]. Given a vector fieldf on Rⁿ, we (implicitly) define the sequence:

xk+1= xk+ αf (xk+1). (28) This scheme corresponds to the operator(Id −αf )⁻¹.

Theorem 10 (Implicit Euler method on WP spaces): Let k · k denote a norm with compatible WP J·, ·K. Let f be a c-strongly contracting vector field with unique equilibrium pointx^∗ and Lipschitz constant `. Then

(i) the(Id −αf )⁻¹ is a contraction mapping with contraction factor(1 + αc)⁻¹ for anyα > 0;

(ii) if α` < 1, then, at each time k, the implicit equation (28) is well-posed and the fixed-point iteration x^[0]_k+1 = xk, x^[i+1]_k+1 = xk+ αf (x^[i]_k+1) is a contraction mapping with contraction factorα`;

(iii) if α` < 1 and kf (x0)k ≤ 2(1+αc)(1−α`)

α(1+α`) , then, at each timek, the Newton-Raphson iteration x^[0]_k+1= xk, x^[i+1]_k+1 = x^[i]_k+1−Dg(x^[i]_k+1)⁻¹(g(x^[i]_k+1)−xk), for g(x) = x − αf (x), converges quadratically to the solution the implicit equation (28).

Proof: Given two sequences {xk}^∞_k=1 and {yk}^∞_k=1 generated by (28), the properties of WPs in IV-Aimply:

kxk+1− yk+1k²

=Jxk− yk+ α(f (xk+1) − f (yk+1)), xk+1− yk+1K

≤Jxk− yk, xk+1− yk+1K

+ αJf (xk+1) − f (yk+1), xk+1− yk+1K

≤ kxk− ykkkxk+1− yk+1k − cαkxk+1− yk+1k².

(6)

Measure Demidovich One-sided Lipschitz

bound condition condition

µ_2,P^1/2(Df (x)) ≤ b P Df (x) + Df (x)^>P 2bP (x − y)^>P f (x) − f (y) ≤ bkx − yk²_P1/2

µ1(Df (x)) ≤ b sign(v)^>Df (x)v ≤ bkvk1 sign(x − y)^>(f (x) − f (y)) ≤ bkx − yk1

µ_∞(Df (x)) ≤ b max

i∈I∞(v)vi(Df (x)v)_i≤ bkvk²_∞ max

i∈I∞(x−y)(xi− yi)(fi(x) − fi(y)) ≤ bkx − yk²_∞ TABLE II: Table of equivalences between measure bounded Jacobians, differential Demidovich and one-sided Lipschitz conditions.

After simple manipulation we obtain kxk+1− yk+1k ≤ (1 + cα)⁻¹kxk−ykk; this proves(i); for a more general treatment see [3]. The proof of statement (ii) is immediate, since the Lipschitz constant of x 7→ x + αf (x) is α`. The proof of statement(iii) relies upon [7, Theorem C] and is omitted in the interest of brevity.

A conjecture is that the Newton-Raphson iteration converges globally and not only locally.

F. Comments on Higher Order Algorithms

We here briefly present the extra-gradient algorithm and prove that it has accelerated convergence over the forward step method. Let f be a vector field on Rⁿ. The extra- gradient iterationswith step size α are given by

xk+0.5 = xk+ αf (xk),

xk+1= xk+ αf (xk+0.5). (29) Theorem 11 (Extra-gradient method on WP spaces):

Let k · k denote a norm with compatible WP J·, ·K. Let f be a c-strongly contracting vector field with unique equilibrium point x^∗, Lipschitz constant `, and condition numberκ = ^`_c ≥ 1. Then

(i) the extra-gradient iterations (29) satisfy kxk+1− x^∗k ≤ 1 + α³`³

1 + αc kxk− x^∗k

and, for every 0 ≤ α ≤ _cκ¹^√_κ, the sequence {xk}^∞_k=0 converges tox^∗;

(ii) for α = _2cκ¹^√_κ, the convergence factor is 1 − 3

8κ√

κ+ O1 κ³

.

The proof of this theorem is omitted in the interest of brevity. It is an open conjecture whether the optimal convergence factor is of order1 − 1/κ.

V. CONCLUSIONS

Contraction theory and monotone operator theory are well established methodologies to tackle control, optimization and learning problems. This article surveys connections among them and shows how to generalize some elements of these theories to Riemannian manifolds and non-Euclidean norms.

REFERENCES

[1] Z. Aminzare and E. D. Sontag. Logarithmic Lipschitz norms and diffusion-induced instability. Nonlinear Analysis: Theory, Methods &

Applications, 83:31–49, 2013. doi:10.1016/j.na.2013.01.

001.

[2] Z. Aminzare and E. D. Sontag. Contraction methods for nonlinear systems: A brief introduction and some open problems. In IEEE Conf.

on Decision and Control, pages 3835–3847, December 2014. doi:

10.1109/CDC.2014.7039986.

[3] P. Cisneros-Velarde and F. Bullo. A contraction theory approach to optimization algorithms from acceleration flows. 2021. URL:https:

//arxiv.org/pdf/1803.08277.pdf.

[4] P. L. Combettes and J.-C. Pesquet. Fixed point strategies in data science. IEEE Transactions on Signal Processing, 2021. doi:10.

1109/TSP.2021.3069677.

[5] J. X. Da Cruz Neto, O. P. Ferreira, and L. R. Lucambio P´erez.

Contributions to the study of monotone vector fields. Acta Math- ematica Hungarica, 94(4):307–320, 2002. doi:10.1023/A:

1015643612729.

[6] A. Davydov, S. Jafarpour, and F. Bullo. Non-Euclidean contraction theory for robust nonlinear stability. IEEE Transactions on Automatic Control, July 2021. Submitted. URL:https://arxiv.org/abs/

2103.12263.

[7] C. A. Desoer and H. Haneda. The measure of a matrix as a tool to analyze computer algorithms for circuit analysis. IEEE Transactions on Circuit Theory, 19(5):480–486, 1972. doi:10.1109/TCT.

1972.1083507.

[8] E. Hairer and G. Wanner. Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems. Springer, 1996. doi:

10.1007/978-3-642-05221-7.

[9] S. Jafarpour, A. Davydov, and F. Bullo. Non-Euclidean contraction theory for monotone and positive systems. IEEE Transactions on Automatic Control, September 2021. URL:https://arxiv.org/

abs/2104.01321.

[10] S. Jafarpour, A. Davydov, A. V. Proskurnikov, and F. Bullo. Robust implicit networks via non-Euclidean contractions. 2021. URL:http:

//arxiv.org/abs/2106.03194.

[11] W. Lohmiller and J.-J. E. Slotine. On contraction analysis for nonlinear systems. Automatica, 34(6):683–696, 1998. doi:10.1016/

S0005-1098(98)00019-3.

[12] S. Z. N´emeth. Geodesic monotone vector fields. Lobachevskii Journal of Mathematics, 5:13–28, 1999. URL:http://mi.mathnet.ru/

eng/ljm145.

[13] M. Revay, R. Wang, and I. R. Manchester. Lipschitz bounded equilibrium networks. 2020. URL:https://arxiv.org/abs/

2010.01732.

[14] E. K. Ryu and S. Boyd. Primer on monotone operator methods.

Applied Computational Mathematics, 15(1):3–43, 2016.

[15] N. K. Sahu, R. N. Mohapatra, C. Nahak, and S. Nanda. Approximation solvability of a class of A-monotone implicit variational inclusion problems in semi-inner product spaces. Applied Mathematics and Computation, 236:109–117, 2014. doi:10.1016/j.amc.2014.

02.095.

[16] J. W. Simpson-Porco and F. Bullo. Contraction theory on Riemannian manifolds. Systems & Control Letters, 65:74–80, 2014. doi:10.

1016/j.sysconle.2013.12.016.

[17] J. H. Wang, G. L´opez, V. Mart´ın-M´arquez, and C. Li. Monotone and accretive vector fields on Riemannian manifolds. Journal of Optimization Theory and Applications, 146(3):691–708, 2010. doi:

10.1007/s10957-010-9688-z.

[18] P. M. Wensing and J.-J. E. Slotine. Beyond convexity — Contraction and global convergence of gradient descent. PLoS One, 15(8):1–29, 2020. doi:10.1371/journal.pone.0236661.