DIGITAL OPTIMAL C O N T R O L S Y S T E M S
WITH S T O C H A S T I C P A R A M E T E R S
DIGITAL O P T I M A L C O N T R O L S Y S T E M S WITH
S T O C H A S T I C P A R A M E T E R S
o ~J BIBLIOTHEEK T U Delft P 2109 5094 C 20672 871238-DIGITAL OPTIMAL C O N T R O L S Y S T E M S
WITH S T O C H A S T I C P A R A M E T E R S
P R O E F S C H R I F T t e r v e r k r i j g i n g v a n d e g r a a d v a n d o c t o r i n d e t e c h n i s c h e w e t e n s c h a p p e n a a n d e T e c h n i s c h e H o g e s c h o o l D e l f t o p g e z a g v a n d e R e c t o r M a g n i f i c u s P r o f . i r . B . P . T h . V e l t m a n i n h e t o p e n b a a r t e v e r d e d i g e n t e n o v e r s t a a n v a n h e t C o l l e g e v a n D e k a n e n o p w o e n s d a g 10 o k t o b e r 1 9 8 4 t e 1 0 . 0 0 u u r d o o r W I L L E M L E O P O L D D E K O N I N G g e b o r e n t e L e i d e n e l e k t r o t e c h n i s c h i n g e n i e u r\i 013714 ij
\»
....*/
D e l f t U n i v e r s i t y P r e s s / 1 9 8 4Dit proefschrift is goedgekeurd door de
promotor Prof.dr. A . J o h n s o n
aan Barbara
aan Lorena
Heeft een hond Boeddha-natuur?
Dit is de meest serieuze vraag van allemaal. Als je ja of nee zegt,
Verlies je je eigen Boeddha-natuur. Z e n m e e s t e r M u m o n , 1 1 8 3 - 1 2 6 0 , C h i n a
PREFACE
T h i s d i s s e r t a t i o n i s an o u t g r o w t h o f my r e s e a r c h a c t i v i t i e s w i t h t h e P r o c e s s Dynamics and C o n t r o l Group o f t h e Department o f A p p l i e d P h y s i c s o f t h e D e l f t U n i v e r s i t y o f T e c h n o l o g y , D e l f t , The N e t h e r l a n d s , d u r i n g the p e r i o d 1978 - 1983. Many p e o p l e have c o n t r i b u t e d t o i t s c o m p l e t i o n . U n f o r t u n a t e l y acknowledgements may n o t be i n s e r t e d h e r e a c c o r d i n g t o u n i v e r s i t y régulations. T h e r e f o r e I w i l l c o n f i n e m y s e l f t o m e n t i o n a few names. A l a n J o h n s o n , my s u p e r v i s i n g p r o f e s s o r , gave me t h e freedom t o a r r a n g e t h i s r e s e a r c h a c c o r d i n g t o my own i n s i g h t s . He d e l i v e r e d i n d i s p e n s a b l e s u p p o r t , as w e l l as v a l u a b l e c r i t i c i s m s and s u g g e s t i o n s .
Hennie P o u l i s s e , J a n Kaan, J a n - J o o s t van Roosmalen, A r t h u r T u j e e h u t , R i n u s van P r a a g , Henk J u n i e r , G e r a r d de G r o o t , Ton van W i n g e r d e n ,
J a n - L o d e w i j k de J o n g and Lex Tiedemann a l l p a r t i c i p a t e d i n t h i s r e s e a r c h i n t h e c o n t e x t o f t h e i r s t u d y . The f i r s t - m e n t i o n e d s t a r t e d t h i s i n an i n s p i r i n g way and f o l l o w e d t h e whole r e s e a r c h c r i t i c a l l y .
The s t a f f o f t h e L a b o r a t o r y f o r P h y s i c a l T e c h n o l o g y , where t h i s r e s e a r c h t o o k p l a c e , made t h e e n v i r o n m e n t p l e a s a n t t o work i n .
CONTENTS
1. INTRODUCTION 1
2. INFINITE HORIZON OPTIMAL CONTROL OF LINEAR
DISCRETE TIME SYSTEMS WITH STOCHASTIC PARAMETERS 3 Automática 18_ (1982) 443-453 2.1. I n t r o d u c t i o n 3 2.2. N o t a t i o n s and m a t h e m a t i c a l p r e l i m i n a r i e s 4 2.3. S t a b i l i z a b i l i t y and o b s e r v a b i l i t y 6 2.4. F i n i t e h o r i z o n o p t i m a l c o n t r o l 8 2.5. I n f i n i t e h o r i z o n o p t i m a l c o n t r o l 9 2.6. C o n d i t i o n s f o r s t a b i l i z a b i l i t y 10 2.7. C o n c l u s i o n s 12 R e f e r e n c e s 12
3. DETECTABILITY OF LINEAR DISCRETE-TIME SYSTEMS WITH STOCHASTIC
PARAMETERS 15 I n t . J . C o n t r o l 38 (1983) 1035-1046 3.1. I n t r o d u c t i o n 15 3.2. N o t a t i o n s and m a t h e m a t i c a l p r e l i m i n a r i e s 16 3.3. D e t e c t a b i l i t y 17 3.4. O p t i m a l c o n t r o l 21 3.5. C o n d i t i o n s f o r d e t e c t a b i l i t y 23 3.6. C o n c l u s i o n s 25 R e f e r e n c e s 26
4. ON VARIANCE NEUTRALITY OF SYSTEMS WITHIN GENERAL
RANDOM PARAMETERS 27 IEEE T r a n s . Autom. C o n t r o l AC-28 (1983) 101-103
4 . 1 . I n t r o d u c t i o n 27 4 . 2 . P r o b l e m f o r m u l a t i o n 27 4 . 3 . S t a t e e s t i m a t i o n and v a r i a n c e n e u t r a l i t y 27 4 . 4 . C o n c l u s i o n s 28 A p p e n d i x 28 R e f e r e n c e s 29
5. OPTIMAL ESTIMATION OF LINEAR DISCRETE TIME
SYSTEMS WITH STOCHASTIC PARAMETERS 31 A u t o m á t i c a £ 0 (1984) 113-115 5 . 1 . I n t r o d u c t i o n ^ 5 . 2 . O p t i m a l e s t i m a t i o n 31 5 . 3 . Time i n v a r i a n t e s t i m a t i o n 32 5 . 4 . C o n c l u s i o n s 33 R e f e r e n c e s 3 3
6. EQUIVALENT DISCRETE OPTIMAL CONTROL PROBLEM
FOR RANDOMLY SAMPLED DIGITAL CONTROL SYSTEMS 35 I n t . J . Systems S e i . 11 (1980) 841-850 6 . 1 . I n t r o d u c t i o n 35 6 . 2 . D i g i t a l o p t i m a l c o n t r o l p r o b l e m 36 6 . 3 . E q u i v a l e n t d i s c r e t e o p t i m a l c o n t r o l p r o b l e m i n t h e c a s e o f d e t e r m i n i s t i c s a m p l i n g 37 6 . 4 . E q u i v a l e n t d i s c r e t e o p t i m a l c o n t r o l p r o b l e m i n t h e c a s e o f S t o c h a s t i c s a m p l i n g 42
6.5. C o n c l u s i o n s R e f e r e n c e s
ON THE OPTIMAL CONTROL OF RANDOMLY SAMPLED LINEAR STOCHASTIC SYSTEMS
P r o c . IFAC Symp. CAD C o n t r o l S y s t . (1980) 107-112 7.1. I n t r o d u c t i o n 7.2. P r o b l e m f o r m u l a t i o n 7.3. E q u i v a l e n t d i s c r e t e o p t i m a l c o n t r o l p r o b l e m 7.4. O p t i m a l d i g i t a l c o n t r o l 7.5. C o n c l u s i o n s R e f e r e n c e s
STATIONARY OPTIMAL CONTROL AND ESTIMATION OF STOCHASTICALLY SAMPLED CONTINUOUS-TIME SYSTEMS S u b m i t t e d f o r p u b l i c a t i o n 7.1. I n t r o d u c t i o n 7.2. O p t i m a l c o n t r o l p r o b l e m 7.3. O p t i m a l c o n t r o l 7.4. O p t i m a l e s t i m a t i o n 7.5. I n f l u e n c e o f s t o c h a s t i c s a m p l i n g 7.6. C o n c l u s i o n s Acknowledgements R e f e r e n c e s
THE INFLUENCE OF FINITE WORD LENGTH ON DIGITAL OPTIMAL CONTROL
IEEE T r a n s . Autom. C o n t r o l AC-29 (1984) 385-391 9.1. I n t r o d u c t i o n
9 . 2 . S t o c h a s t i c s o f r o u n d o f f e r r o r s 88 9 . 3 . D é r i v a t i o n o f means and v a r i a n c e s 89 9 . 4 . Roundoff e r r o r s i n o p t i m a l l y c o n t r o l l e d Systems 91 9 . 5 . C o n c l u s i o n s 92 Appendix 92 R é f é r e n c e s 92
b-10. THE EQUIVALENT DISCRETE-TIME OPTIMAL CONTROL PROBLEM FOR CONTINUOUS-TIME SYSTEMS
WITH STOCHASTIC PARAMETERS 95 To appear i n I n t . J . C o n t r o l 1 0 . 1 . I n t r o d u c t i o n 96 1 0 . 2 . D i g i t a l o p t i m a l c o n t r o l p r o b l e m 98 1 0 . 3 . E q u i v a l e n t d i s c r e t e - t i m e System 101 1 0 . 4 . E q u i v a l e n t d i s c r e t e - t i m e o p t i m a l c o n t r o l p r o b l e m 116 1 0 . 5 . C o n c l u s i o n s 123 A p p e n d i x 124 Acknowledgements 126 R é f é r e n c e s 127
11. STATISTICAL AND STABILIZABILITY PROPERTIES OF EQUIVALENT
DISCRETE-TIME SYSTEMS WITH STOCHASTIC PARAMETERS 131 To a p p e a r i n I n t . J . C o n t r o l 1 1 . 1 . I n t r o d u c t i o n 132 1 1 . 2 . S c a l a r é q u i v a l e n t d i s c r e t e - t i m e System 134 1 1 . 3 . C o r r é l a t i o n 137 1 1 . 4 . S t a b i l i z a b i l i t y 140 1 1 . 5 . C o n c l u s i o n s 147 R é f é r e n c e s 148 SUMMARY 151 SAMENVATTING 153
CURRICULUM VITAE
1. INTRODUCTION An i m p o r t a n t c l a s s o f s a m p l e d - d a t a Systems a r e d i g i t a l c o n t r o l S y s t e m s , i . e . where a c o n t i n u o u s - t i m e s y s t e m i s c o n t r o l l e d b y a d i g i t a l computer. S t o c h a s t i c p a r a m e t e r s may a r i s e i n d i g i t a l c o n t r o l Systems m a i n l y because o f t h r e e r e a s o n s . F i r s t l y t h e s a m p l i n g p r o c e s s may be s t o c h a s t i c , i n t e n t i o n a l o r u n i n t e n t i o n a l . S e c o n d l y t h e f i n i t e word l e n g t h o f t h e computer i n t r o d u c e s s t o c h a s t i c r o u n d i n g e r r o r s i n t h e a r i t h m e t i c . T h i r d l y t h e c o n t i n u o u s - t i m e System may have s t o c h a s t i c p a r a m e t e r s . U s u a l l y thèse phenomena a r e n o t t a k e n i n t o a c c o u n t i n t h e détermination o f t h e o p t i m a l C o n t r o l l e r . However i f t h i s C o n t r o l l e r i s a p p l i e d , t h e n t h e t r u e c o n t r o l System, i . e . w i t h s t o c h a s t i c p a r a m e t e r s , may have a s e r i o u s l y degraded c o n t r o l p e r f o r m a n c e o r may even be u n s t a b l e i f t h e P a r a m e t e r v a r i a t i o n s a r e r e l a t i v e l y l a r g e . R e p r e s e n t i n g t h e s t o c h a s t i c p a r a m e t e r s by a d d i t i v e n o i s e i n t h e détermination o f t h e o p t i m a l C o n t r o l l e r would n o t h e l p much, s i n c e , u n l i k e s t o c h a s t i c p a r a m e t e r s , a d d i t i v e n o i s e c a n n o t d e s t a b i l i z e an o p t i m a l c o n t r o l System. I f t h e pararn.eter v a r i a t i o n s a r e r e l a t i v e l y l a r g e on e m i g h t say t h a t i n t h e c o n t e x t o f o p t i m a l c o n t r o l an e s s e n t i a l p a r t o f r e a l i t y , w h i c h c a n n o t be r e p r e s e n t e d b y a d d i t i v e n o i s e , can be r e p r e s e n t e d by s t o c h a s t i c p a r a m e t e r s . T h i s t h e s i s e x t e n d s t h e e x i s t i n g t h e o r y o f d i g i t a l o p t i m a l c o n t r o l Systems w i t h d e t e r m i n i s t i c p a r a n e t e r s t o d i g i t a l o p t i m a l c o n t r o l Systems w i t h s t o c h a s t i c p a r a m e t e r s , where l i n e a r S y s t e m s , q u a d r a t i c c r i t e r i a and w h i t e s t o c h a s t i c p a r a m e t e r s a r e a l w a y s assumed. O p t i m a l c o n t r o l o f Systems w i t h s t o c h a s t i c p a r a m e t e r s a p p e a r s t o be e s s e n t i a l l y différent from o p t i m a l c o n t r o l o f Systems w i t h d e t e r m i n i s t i c p a r a m e t e r s . T h i s i s r e f l e c t e d i n a number o f ways. F i r s t l y t h e o p t i m a l c o n t r o l o f Systems w i t h d e t e r m i n i s t i c p a r a m e t e r s c a n be s t u d i e d i n t h e main t h r o u g h t h e b e h a v i o u r o f t h e f i r s t moments o f t h e r e l e v a n t v a r i a b l e s . I f t h e p a r a m e t e r s
a r e s t o c h a s t i c t h i s i s no l o n g e r t h e c a s e . Then t h e s e c o n d moments o f t h e v a r i a b l e s p l a y a d e f i n i t i v e r ô l e . S e c o n d l y t h e d u a l i t y between c o n t r o l and s t a t e e s t i m a t i o n , which e x i s t s i n t h e case o f d e t e r m i n i s t i c p a r a m e t e r s , i s l o s t . T h i r d l y t h e s é p a r a t i o n o f c o n t r o l and s t a t e e s t i m a t i o n i s p o s s i b l e i n the c a s e o f d e t e r m i n i s t i c p a r a m e t e r s b u t n o t i n t h e c a s e o f s t o c h a s t i c p a r a m e t e r s . F o u r t h l y t h e u s u a l n o t i o n s o f s t a b i l i z a b i l i t y and d e t e c t a b i l i t y a r e no l o n g e r s u i t a b l e . They have t o be g e n e r a l i z e d t o t h e n o t i o n s o f s o -c a l l e d mean s q u a r e s t a b i l i z a b i l i t y and mean s q u a r e d e t e -c t a b i l i t y , e m p h a s i z i n g t h e g e n e r a l i z a t i o n o f t h e f i r s t moments a p p r o a c h t o t h e second moments
a p p r o a c h a s m e n t i o n e d a b o v e .
A f r u i t f u l s t a r t i n g p o i n t f o r t h e s t u d y o f d i g i t a l c o n t r o l Systems w i t h s t o c h a s t i c p a r a m e t e r s a p p e a r s t o be t h e s t u d y o f d i s c r e t e - t i m e Systems w i t h s t o c h a s t i c p a r a m e t e r s , which a r e a l s o i m p o r t a n t i n t h e i r own r i g h t . A r e a s o f a p p l i c a t i o n f o r which t h e a n a l y s i s o f d i s c r e t e - t i m e Systems w i t h s t o c h a s t i c p a r a m e t e r s might be u s e f u l a r e , f o r e x a m p l e , economic Systems and a n a l y t i c a l c h e m i s t r y .
T h i s t h e s i s c o n s i s t s e s s e n t i a l l y o f f o u r p a r t s . The f i r s t p a r t , c h a p t e r 2, 3 , 4 and 5, c o n s i d e r d i s c r e t e - t i m e Systems w i t h s t o c h a s t i c p a r a m e t e r s , and i s t h e b a s i s f o r t h e o t h e r p a r t s . The second p a r t , c h a p t e r 6, 7 and 8 , i n v e s t i g a t e s d i g i t a l o p t i m a l c o n t r o l Systems where t h e s a m p l i n g p r o c e s s i s s t o c h a s t i c . The t h i r d p a r t , c h a p t e r 9 , s t u d i e s t h e e f f e c t o f t h e f i n i t e word l e n g h t on d i g i t a l o p t i m a l c o n t r o l Systems. The f o u r t h p a r t , c h a p t e r 10 and 1 1 , c o n s i d e r s d i g i t a l o p t i m a l c o n t r o l Systems where t h e c o n t i n u o u s - t i m e System has
s t o c h a s t i c p a r a m e t e r s .
The t h e s i s c o n s i s t s o f a c o l l e c t i o n o f t e n p a p e r s which
have been o r w i l l be p u b l i s h e d i n t h e l i t e r a t u r e . They a i l a r e i n s e r t e d here as p u b l i s h e d o r s u b m i t t e d .
Automatka. Vol- 18, No. 4, pp. 443-453, 1982
Printed in Great Britain. 0OO5-1098/82/O4O443-I l$03.00/0 Pergamon Press Ltd. © 1982 International Fédération of Automatic Control
Infinite Horizon Optimal Control of Linear
Discrète Time Systems with Stochastic
Parameters*
W. L . D E K O N I N G tBy generalization of the usual notions of stabilizability and observability it is possible to extend the solution of the infinite horizon optimal control problem for Systems with deterministic parameters to the practically more important case of Systems with stochastic parameters, where the existence of the solution may be tested by means of a simple explicit condition in the System matrices.
Key Words—Optimal control; discrète time Systems; stochastic parameters; stabilizability; obser-vability; infinite horizon control.
Abstract—The infinite horizon optimal control problem is considered i n the gênerai case of linear discrète time Sys-tems and quadratic criteria, both with stochastic parameters which are independent with respect to time. A stronger stabilizability property and a weaker observability property than usual for deterministic Systems are introduced. It is shown that the infinite horizon problem has a solution if the System has the first property. If in addition the problem has the second property the solution is unique and the control System is stable in the mean square sensé. A simple neces-sary and sufficient condition, explicit in the System matrices, is given for the System to have the stronger stabilizability property. This condition also holds for deterministic Systems to be stabilizable in the usual sensé. The stronger stabil-izability and weaker observability properties coïncide with the usual ones if the parameters are deterministic.
1. I N T R O D U C T I O N
UN C E R T A I N T I E S concerning a plant are most often represented by additive noise. If the Parameter variations are large, then représen-tation of the uncertainties by multiplicative noise or more generally by stochastic parameters is more realistic. Also digital control Systems where the sampling occurs stochastic-ally give rise to Systems with stochastic parameters.
Areas of applications for which the analysis of discrète time Systems with stochastic parameters might be useful, are for example: digital control of chemical processes; Systems
*Received 19 June 1981; revised 26 October 1981; revised 3 December 1981. The original version of this paper was not presented at any I F A C meeting. This paper was recom-mended for publication in revised form by associate editor T. Ba§ar under the direction of editor H . Kwakernaak.
^Department of Applied Physics, Delft University of Technology, Delft, The Netherlands.
with human Operators; economic Systems; and stochastically sampled digital control Systems. The first three areas are most naturally modelled by discrète time Systems where the uncertainty of the related parameters is in-herently large. The last mentioned application may arise in cases where, intentional or unin-tentional, stochastic sampling causes the parameters of the corresponding discrète time System, and perhaps the criterion, to be sto-chastic.
It is well known (Kwakernaak and Sivan, 1972) that the usual infinite horizon L Q problem for a deterministic System has a solution if the System is stabilizable. If in addition the problem is observable the solution is also unique and the control System is asymptotically stable. It ap-pears that in the case of Systems with stochastic parameters the infinite horizon problem does not always have a solution if the deterministic version of the problem, i.e. the parameters replaced by their mean values, has a solution. The question therefore arises as to the con-ditions on the Statistical properties of the Sys-tem matrices for which the infinite horizon problem has a solution.
The following will be concerned with linear discrète time Systems and quadratic criteria both having stochastic parameters which are independent with respect to time. For succinct-ness thèse will be referred to as independent stochastic parameters.
The study of the infinite horizon optimal con-trol of linear discrète time Systems with in-dependent stochastic parameters goes back to
444 W. L . D E K O N I N G
Kaiman (1961). He studied a cost criterion without a control term and found a necessary and sufficient condition for which the solution of the infinite horizon problem exists. However, this condition is not explicit in the System matrices and must be calculated by successive évaluations of a discrète time matrix Riccati équation. Drenick and Shaw (1964) succeeded in finding an explicit necessary and sufficient con-dition for the existence of the solution of the infinite horizon problem in the case of a second order system with scalar input where the Parameters are not mutually correlated. Katay-ama (1976) considered a deterministic system where the control matrix is multiplied by scalar, independent noise. He found an explicit neces-sary and sufficient condition for the existence of the solution of the infinite horizon problem. K u and Äthans (1977) generalized the resuit of Katayama to a multivariable deterministic Sys-tem with a nonsingular control matrix where both the system and control matrix are multi-plied by scalar independent noise. They also found an explicit necessary and sufficient con-dition for the existence of the solution of the infinite horizon problem. Related results are also given by Connors (1967) and Harris (1978) who define various forms of stochastic controllability for Systems with independent stochastic Parameters.
In this paper a substantial contribution to the solution of the infinite horizon optimal control problem in the general case of linear discrète time Systems and quadratic criteria both with stochastic parameters is made. The only re-striction on the system and criterion parameters is that they are independent in time. We show that in the case of stochastic parameters the system must have a stronger stabilizability pro-perty than the usual one for the existence of a solution of the infinite horizon optimal control problem. It will also be shown that in addition the problem need have a weaker observability property than the usual one for the uniqueness of the solution and the stability in the mean square s e n s é of the control system. Because of the crucial role of stabilizability in the solution of the infinite horizon problem, it is important to find conditions on the Statistical properties of the system matrices for which the system has the désirable property of stabilizability. This paper gives a simple necessary and sufficient condition, explicit in the system matrices, for Systems with independent stochastic parameters to have the stronger stabilizability property. Our property of stabilizability coincides with stabil-izability in the usual s e n s é if the system is deterministic. That means that the above
con-dition also holds for deterministic systems to be stabilizable in the usual sense. This condition is illustrated with some examples.
Concerning observability, it is shown that the optimal control problem with stochastic parameters possesses the weaker observability property if the deterministic versión of the problem, i.e. the parameters replaced by their mean valúes, is observable in the usual sense. This explains why in the literature observability problems never arise. There it is always assumed that the deterministic versión of the control problem is observable in the usual sense. Our property of observability coincides with observability in the usual sense if the con-trol problem is deterministic.
The optimal control of linear discrete time systems with deterministic parameters and quadratic criteria can be studied in the main through the behaviour of the first moments of the relevant variables. If the parameters are stochastic this is no longer the case. Then the second moments of the variables play a definitive role in the optimal control problem. The theory of monotonic transformations on finite dimensional Banach spaces with a partial ordering appears to be a key tool to transíate the standard 'first moments' approach for systems with deterministic parameters to the more general 'second moments' approach for systems with stochastic parameters.
2. N O T A T I O N S A N D M A T H E M A T I C A L P R E L I M I N A R I E S
In this section some notations and basic mathematical facts are discussed which will be needed later. In particular some results are stated concerning monotonic transformations and a generalized discrete time Lyapunov equation.
Scalars are denoted by lower case Greek let-ters, column vectors by lower case italic letlet-ters, matrices by capital Greek or italic letters, transformations and spaces by capital script letters. The symbols i, j, k, I, m, n, N denote nonnegative integers. Column vectors are viewed as one column matrices. The transpose is denoted by superscript T. Suppose a,a¡ are elements of an arbitrary space with norm |||| then lim II || = 0 is denoted by a¡ -* a and the sequence a0, a . . . by {a,}.
3?" denotes the Euclidean n-space with inner product (x, y) = xTy, x, y £ 9t" and norm ||x|| =
(xTx)"2, x £ £5?". The zero element is denoted by
0. Mm" denotes the Banach space of all real
m x n matrices with norm llAll = max||Ax|l, A £
Mm", x £ 3 T .
Infinité horizon control of Systems with stochastic parameters 445
The zero and identity element are denoted by 0 and I. M"" is denoted by M". The spectrum of A G M" is denoted by o-(A), the spectral radius by p(A) and A is called stable if p(A)< 1. if" d é n o t e s the Banach space of ail real symmetrie
n x n matrices with the same norm as M". The
matrix A G if" is called nonnegative definite, denoted by AarO, if xTA i > 0 , V x G $ " , and
positive definite, denoted by A > 0, if in addition xTA x = 0 x = 0. Suppose A , A, G if" then {A,}
is called bounded if 3 B G if" 3 Af £ B and
in-creasing if i a j A, a At. ST" d é n o t e s the Banach space of ail linear transformations
si: if" ^if" with norm \\si\\ = max \AX\, X G
lixl-i y".
The zero and identity element are denoted by
Û and $. The spectrum of si G 3~" is denoted by cr(si), the spectral radius by p(si) and ^ is
called stable if p(stl) < 1.
The properties of . « " and ST" are very similar and well known (Kreyszig, 1978). Only the pro-perties that for a G W or JC" p(a) = lim ||a'||"' s Il<3II and that for a S if" p( a ) = ||a|| are mentioned.
Furthermore it is mentioned that {Ai a 0} boun-ded and increasing => A, -» A, A > 0, Ai < A .
An arbitrary transformation (not necessarily linear) si: if" -» if" is called monotonie (Kras-nosel'skii, 1964) if X , Y G if", 0 < X <
Y^siXssiY. Concerning monotonie
trans-formations the following lemma is stated.
Lemma 2.1. Suppose the arbitrary
transfor-mation si : if" -> if" and X G if" then
(a) si monotonie and i X > 0 , VXaO=>.stf' monotonie and si 'X a 0, V X a 0, Vi ;
(b) if si G 3"" then .stf monotonie O (X a 0 4 > ^ X > 0 ) ;
(c) if si E. if" and monotonie then | | ^ J | = \\si\\,
si' G ¡7" and monotonie, V i , and (si
stable ^ ' X 0, X > 0).
Proof. Proofs are simple and left to the
reader except the part of the last statement of (c). X > 0 ^ 3 a > 0 3 a X > / a n d because si monotonic, si'I s si'aX. Now p'(si) = p(si') s
Therefore | | ^ ' X | 0 si stable. • With the aid of monotonic transformations we
can state the following lemma about a general-ized discrete time Lyapunov équation.
Lemma 2.2. Consider the équation
X = siX + B, si G ¿7" and monotonie, B a 0 (1)
then
(a) si stable 3 solution X a 0.
(b) 3 solution X a O , B > 0 => .stf stable, X > 0.
Proof: We first prove part (a). Define X =
2 si'B. Then si stable X exists and from the
i=0
définition X a O . Furthermore
X - siX = Y si'B - Y ^jB = B,
thus X is a solution. Now prove part (b). From (1) one has by induction
The right-hand summation is increasing and bounded, thus 2 sikB exists and therefore
sikB->6. Then by Lemma 2.1c si is stable. Furthermore
i-i
X a ^ j si"B a B > 0.
The Moore-Penrose pseudo-inverse of the matrix AEMm" is denoted by A * and defined by A+A x = x, Vx G »(AT) and A+x = 0, Vx G
M(AT) where and JV(-) dénote range space and null space, respectively. I mention only the properties (Barnett, 1971) that A" is unique, A+ = A- 1 if A is nonsingular, A A * A = A and
A* G ¿7" if A G if".
Suppose the matrix A G M'"" then as and at dénote respectively the element in the ith row, jth column and the ith column. A may be denoted by (a/;) or (a/). The stack of A is denoted by st (A) and defined by the column vector (af arn)r. The dimension of st (A) is mn. The Kronecker product of the matrices A G Mm", B G Mm'"' is denoted by A ® B and defined by (a/jB). The dimension of A ® B is mm'+ fin'. Suppose that d é n o t e s the element of A ® B in the yth row, A/th column where the pairs ij and kl are ordered as in st (A) then y,7 W = aj/fla. We mention only the properties
(Bellman, 1970) that for A G B G M'm, C G
.^T" st (ABC) = ( CT ® A)st (B), (A ® C )T =
AT® CT.
The expectation of a stochastic variable a is
denoted by E {a} or a. The probability of an event a is denoted by P{a}. For the random matrices Ai, B„ where the statistics are in-dependent of i, Âf is denoted by  and the pair
446 W . L . D E K O N I N G
moments of a stochastic variable exist and one uses the abbreviation m for mean and ms for mean square. Finally, the reader should recall the Lebesgue monotone convergence theorem (Burrill, 1972), i.e. if {a,} is an increasing sequence of nonnegative random variables and
a,-* a a.s. then di -» â.
3. S T A B I L I Z A B I L I T Y A N D O B S E R V A B I L I T Y
In this section generalizations of the usual properties of stabilizability and observability are introduced for linear discrete time systems with independent stochastic parameters. Also, necessary and sufficient conditions for these properties are derived. At the end of the section we state a lemma concerning a generalized dis-crete time Lyapunov equation and the general-ized observability property.
Consider the open loop system
x ,+, - * f t + r/Hi (2)
where x, e 3Î" is the state, w, G 9Lm the control, and «ti G M", Ti £ M"m are the system matrices. The processes {<Pf} and {I^} are sequences of
independent random matrices with constant statistics. For convenience the initial time is i = 0. The initial value x0 is deterministic. It is
assumed that Ui is a deterministic function of xf,
j £ i. Then <J>i and r, are independent of x„ j s i. System (2) is characterized by the pair (<J>, T). Suppose
u, = - Lxi (3) where L e i " and define *L, i by
¥W= . * , - T , L (4)
then from (2) the closed loop system is
Theorem 3.1. siL G 3~" and monotonie, Vi. Proof: From (6) slL(aX + ßY) = ¥ [ ( a X +
ßY)VL = aVlXVl+ßVTLYVL = as4LX + ßsiLY, V X , Y G <f". Va, ßS®. Thus stL G 3~*. Further-more X > 0 • $ * u j X VU i> 0 ^ . x^ljXVL,,x a 0,
V x e 3 r = > xT¥ l X ¥Lx > 0 , VxG3?"=>
V\XVL a 0, i.e. ,s/L monotonie. Then by
Lemma 2.1c the theorem follows. • Furthermore a lemma is stated which will be
needed a few times in the sequel.
Lemma 3.1. xT,Yx, = xlsilYx* V V ê £ _
Proof: From (5), (6) and by induction x] Yxt =
xlMYV^-y = xl^Yx,., = xlstLYx«, V K G
Now one has the following necessary and
sufficient stability conditions. •
Theorem 3.2. The closed loop system (5) is
m-stable O stable and ms-stable » siL stable.
Proof. From (5) by induction Xl—9tßlM =
WLx0. Then x,->0, Vx0 O -» 0 • » ¥L stable.
This proves the first part. From Lemma 3.1 with
Y = I one has ||exj2 = x\x, = x'„si'LIxo. Then
H p - > 0 , V x o O x J ^ i J x o ^ O , V x o O ^ i / ^
0 ^L stable. •
The stabilizability definition for the open loop system (2) is as follows.
Definition 3.2. (4>, T) is called m-stabilizable if
3Z. 3 closed loop system (5) is m-stable and ms-stabilizable if 3L 3 closed loop system (5) is ms-stable. Then one has the following necessary and sufficient stabilizability conditions.
Theorem 3.3. (#, O is m - s t a b i l i z a b l e o 3 L 3
#(. stable and ms-stabilizable » 3 L 3 sdL stable.
Proof. Immediate from Definition 3.2 and
Theorem 3.2.
Finally a link is established between mean, mean square stability and stabilizability.
xi +, = * L . ^ . (5)
The following definition concerning stability is stated.
Definition 3.1. The closed loop system (5) is
called m-stable if x , - » 0 , V x0 and ms-stable if
H P - O , Vx„.
To describe the mean square behaviour of x, one needs the transformation siL: ¡7" ^* !f" defined by
siLX = ¥ [ X ¥L, X G ST. (6)
First one has a theorem concerning the linearity and monotonicity of s4L.
Theorem 3.4. For the closed loop system (5)
ms-stability =^ m-stability and for the pair (<P, T) ms-stabilizability => m-stabilizability. If the second moments of <I>i and T, are zero then stability coincides with m-stability and ms-stabilizability with m-ms-stabilizability.
Proof. Because 0 < ||xi||2 £ jfxj5 and the fact
that the second equality is attained when the second moments of <P* and T, are zero, the theorem follows from Definition 3.1 and
Definition 3.2. • ms-Stability and ms-stabilizability are
generalizations of respectively m-stability and m-stabilizability. The definitions of m-stability and m-stabilizability are the same as the definitions of asymptotic stability and stabil-6
Infinité horizon control of Systems with stochastic parameters 447
izability for the deterministic System (<J>, T). ms-Stability is a stronger property than m-stability and ms-stabilizability a stronger property than m-stabilizability.
Now attention is given to observabiiity. Con-sider the system
xl + 1 = *,jci (7a)
yi = (7b) where x^Edl" is the state, y, £ 9 ! ' the
obser-vation and ' t j e i " , Ci EM'" are the system matrices. The processes {Oj} and {d} are s é q u e n c e s of independent random matrices with constant statistics. For convenience the initial time is i = 0. The initial value x0 is deterministic.
One can see that <J>, and C< are independent of x,,
j s ;'. System (7) is characterized by the pair
(<ï>, C). The following définition concerning observabiiity is stated.
Definition 3.3. (<t>, C) is called m-observable if 3k 3 y, = 0, i = 0, 1,. . . , k - 1 x„ = 0 and
ms-observable if 3 f c 3 i | ^ f = 0, 1 = 0,
k - 1 ^ x„ = 0.
It is easy to see that m-observabiiity means that it is possible to reconstruct x0 from y„
i = 0 k - 1 for some k and that
ms-obser-vability means that it is possible to reconstruct x0 from ||y,-||2, i = 0 , . . . , k - 1 for some k. The
following necessary and sufficiënt observabiiity conditions are available.
Theoremi.5. (_<J>, C)Js m-observable O 3k 3
rank of ( CT <Ï>TCT . . . *T t ,CT)T is n and
ms-k-1
observable <a> 3k 3 2 .tfJCTC > 0.
Proof: From (7) we have by induction
y, = Cx> = CÔ'xo._ Then y, =0, i = 0 , , . . , k-1 *> C*^x0 = 0, i = 0 , . . . . A - 1 »
( CT $ CT . . . <I>T'_'CT)T*o = 0. The last équation
implies i0 = 0 o t h e matrix in the last équation
has rank n. From Lemma 3.1 with Y = CTC and
using stL with L = 0 NI* = yTy, = x T cTC x , = xW>cTCx0. Then
W = o,i = o k - i o 2 h F - o
¡=0 < * * « ( J £ ^ i cTc ) x„ = o.The last equation implies x0 = 0 o t h e
expres-sion in parentheses is positive definite. •
Also this gives a resuit concerning the relation-ship between mean and mean square obser-vabiiity.
Theorem 3.6. For (<!>, C)
m-obser-vability ms-obserm-obser-vability. If the second moments of <J>, and T, are zero then ms-obser-vability coincides with m-observabiiity.
Proof: Because 0 s \\y,f ~ ||y,|p and the fact
that the second equality is attained when the second moments of 4>, and T, are zero, the
theorem follows from Définition 3.3. • ms-Observability is a generalization of
m-observability. The définition of m-observabiiity is the same as the usual définition of obser-vabiiity for the deterministic system (<ï>, C). ms-Observability is a weaker property than m-observability. An obvious example of a system which is ms-observable but not m-observable is if one chooses / = n, <P| = I, C, = aj and <*> an independent stochastic variable with â = 0 and ?*<),
Finally in this section a lemma concerning a generalized discrete time Lyapunov equation and ms-observability is stated.
Lemma 3.2. Consider the transformation si:îf"^<f" defined by
dX = ATX A , A random (8)
and the equation
X = dX + B, B random. B > 0 (9)
then 3 Solution X > 0, (A, B "2)
ms-observable => si stable, X > 0.
Proof. In the proof use that by Theorem 3.1, d'E 3~" and monotonic, Vi. (A, B1'2)
ms-observable => 3k 3 2 J 4 ' B > 0 . Because a
solu-¡=0
tion X exists, one has from (9) by induction
k-1
X = dkX + Y .sTB.
1-0
For this equation there exists a Solution X > 0,
k-I ~Zd'B>0
and £ Sf" and monotonic. Thus by Lemma 2.2b sik, and therefore si, is stable and X >0.
•
This lemma is used in Section 5 to prove a stability and a uniqueness property of the optimal control system.
448 W. L . D E K O N I N G
4. F I N I T E H O R I Z O N O P T I M A L C O N T R O L
In this section the finite horizon optimal con-trol problem and its solution for linear discrète time Systems and quadratic criteria, both with independent stochastic parameters is stated. Also stated are some results concerning certain non-linear transformations and a resuit concerning ms-observability of closed loop S y s t e m s .
Consider the System, respectively cost cri-terion
équation may be written as
Js(UN, x0) = xl [g siUQ + LTRL) + siLH j Xo. From the définition of 3ÜL it is seen by induction that 58 lH equals the expression in square
brackets. • The transformation 38L is nonlinear, however,
one has the following resuit. x,+, = + r ,U i (10)
JN(VN, x„) = E |S (xTQiX, + uTR,Ui) + xJ,HxN}
( H )
where x,,E 91 " is the State, u, 6 9?m the control,
<bt&M", r , G j « "m are the System matrices and
Q, e i ' , R , e i ' , H G M" the criterion matrices. The processes {<!>,}, {!",}, {Q,}, {R,} are s é q u e n c e s of independent random matrices with known constant statistics. For convenience the initial time is i = 0. The initial value x0 is
deterministic. Furthermore Q > 0 , R > 0 , H > 0 and UN dénotes the s é q u e n c e {u,, 0 s i < N - 1}. It is assumed that u, is a deterministic function of xh j s i. Then <l\, Th Q, and R, are in-dependent of Xj, j s i. So (11) may also be writ-ten as
JN(UN, x0) = E [g (xTQXi + u]RUi) + xlHxN}. (12) First we State a theorem concerning JN for a special control s é q u e n c e .
Theorem 4.1. Suppose
u, = -Lx, (13)
where LeMmn then
M r jN, x „ ) = xïâ9î!Hxo, Vx0 (14)
where the transformation BL: is defined by
3BLX = siLX + Q + LTR L , X e f (15)
and ,s0L: ST -» if" is defined by (6).
Proof: With (13) the criterion (12) becomes
Js(UN, xo) = E |2 xT(Q + LTR L ) x , + x j H xN J.
From Lemma 3.1 with Y = Q + LTR L this
Theorem 4.2. 35 [ X 2 0, 38 /, monotonie, V X >
0, Vi.
Proo/: From (15) one has X > 0 ^ 38LX >0
and 0 < X £ Y 3BLX s 53LY, i.e. 38L
mono-tonie. Then by Lemma 2.1a the theorem
fol-lows. • Now the définition of the finite horizon
opti-mal control problem is stated.
Definition 4.1. Assume that Uj is a
deter-ministic function of x0,. . . , x„ i = 0,. . . , N - 1,
and given System (11) and criterion (12), then the problem of finding the control s é q u e n c e
[/* = {«*, i = 0,. • . , N— 1} which minimizes JN(UN, XO) for ail x0 and of finding the minimal
value J?S(xo) is called the finite horizon optimal control problem.
In the next theorem the solution of this prob-lem is stated.
Theorem 4.3. The solution of the finite
horizon optimal control problem is given by
u*t = - L£;'-'"x„ i = 0 , . . . , N - l (16)
J £ ( x0) = xÏ38ÏHx„, Vx„ (17)
where the transformation 38*: S?"-*if" is defined by
38*X = 38L xX, X E f (18)
the transformation SBLj[: if" ->if" by (15) and Lx by
Lx = ( rTX r + R ) TTX * , XE.if". (19)
In addition one may write 38 „ X as 38 * X = (pTX<I> - 4 >Tx r ( TTx r + R ) TTX < P
+ 0, x e if". (20)
Proof: The proof is easy. The reader is
refer-red to Aoki (1967) for the necessary
manipula-tions. • Equation (18) is a generalization of the
Infinite horizon control of Systems with stochastic parameters 44<y iar discrete time Riccati équation. Note that in
gênerai 38 * X * 38i.xX, i > 2 . The transformation
38 * is nonlinear but this gives the following result.
Theorem 4.4.
(a) 0 s S 8 * H £ 3 8 £ H , V L , V H a O , V N . (b) 38* monotonic, V N .
Proof: First prove (a). Because O s J ' ( i [ ) s
JN( UN, x0) (14) and (17) gives that 0 <
X J 3 8 * H X0S X03 8 L H X O , V X0. Thus (a) follows.
Consider (b). From (a) one has X = 0 => 38 * X a 0. Furthermore, using Theorem 4.2, 0 < X < y ^ a 9 » x = s 8tx « »t >y = aBxy, i.e. ss*
monotonie. Then by Lemma 2.1 a (b) follows. • Finally in this section a result concerning ms-observability of closed loop Systems is stated. First the following lemma is given.
Lemma 4.1. R>0, 3 8 £ 0 > O v Q > O = >
38?0>0, V L , V/V.
Proof: This lemma i s proved partly b y
prov-ing that R > 0 , j c j » £ e x0 = 0 => x j 3 ä l 0 xo = 0.
From Theorem 4.1 one has XJ33L0XO =
E { 2 (xTQx, + UTR«,)}, «i = - Lxi, i = 0 N - l . Thus xj38?0xo = O, R > 0 4 > u , =
- L x , = 0 a.s., i = 0 N - l=>xj88?0xo =
xÏ38^0Xo = O. Finally O > O ^ 3 8 j ' 0 =
2 s4L(Q + LTR L ) a Q>0. •
(=0
The next theorem relates ms-observability of a closed loop S y s t e m to ms-observability of an open loop System.
Theorem 4.5. R > 0 , (<î>, Q1'2) ms-observable
v Q > 0 OP,., (Q + LTR L ) "2) ms-observable,
where ÍS defined by (4).
Proof: (<t>, Q"2) ms-observable 3 N 3
N - l
2 •stfJQ = 38^0 >0. Then by Lemma 4.1 and i-0
R > 0 or Q > 0 3 N 3 3 8 Î ! 0 = 2'a4[(Q + LTR L ) > 0 and thus ( ^ L , (Q +
i=0
LTR L ) "2) ms-observable. •
In the next section this theorem together with Lemma 3.2 is used to prove a stability and a uniqueness property of the optimal control system.
5. I N F I N I T E H O R I Z O N O P T I M A L C O N T R O L
In this section the infinite horizon optimal control problem for linear discrete time Systems and quadratic criteria, both with independent stochastic parameters, is stated and solved. We also show the crucial rôle of ms-stabilizability.
Consider system (10) and the criterion J4U>, *>) = E jj; (xTQjXi + ulRiU,)} (21) which is the same criterion as (11) except that N is replaced by » and H = 0. The matrices Qj and R, are independent of x,, j < i, so by the Lebes-gue monotone convergence theorem (21) may also be written as
J4U„ x„) = E [ j | (xTQx, + u Ï R u i ) ] . (22)
Now we state the définition of the infinite horizon optimal control problem.
Definition 5.1. Assume that ut is a deter-ministic function of x0, . . . , xh i = 0 , 1 , . . . , and
given system (10) and criterion (21), then the problem of finding the control séquence Ui = {uf} which minimizes JJiU^, x0) for ail x0 and of
finding the minimal value i ï ( x0) is called the
infinite horizon optimal control problem. First consider the limiting properties of 38*0 defined by (18).
Theorem 5.1. (<J>, V) ms-stabilizable 4> S =
lim 38*0 exists, S is the minimal nonnegative definite solution of the équation S = 39* S.
Proof: In the proof Theorem 4.4 is used.
Because 38*0 a 0 and 38* monotonie, one has 38* + l0 = 35*33*0 a 38*0, thus {38*0} increasing.
Also 38*0 a 0, V N . Furthermore because (<t>, T) ms-stabilizable, 3 L 3 iL stable. Then by
Lemma 2.2a the équation X = 38(X =
MLX + Q + LTR L has a solution X a 0. But
using induction X = 38"X a 38^0 thus {38*0} is bounded. Now one has {38*0} is increasing and bounded and 38*0 a 0, V N . Then S = lim 38*0 < °° and S a 0. Because äBj+'ö = 38*ä8£0 one has, taking the limits, S = 38 *S. Suppose S a O is any other solution of S = 38*S. Then 38*0 s 38*S =
S, Vi. Thus, taking the limits, S s S. • The équation 5 = 38* S is a generalization of
the familiär discrete time algebraic Riccati équation. Note that by induction S=38*S = 3ä'L sS, Vi.
In fact it is proved in Theorem 5.1 that, with
H = 0, lim xj38j0x„ = lim J?j(x0) = xJSx0, Vx„. 1t
does not necessarily follow that J î ( x0) = lim
450 W . L . D E K O N I N G
Theorem 5.2. Assume that S = lim 33*^0
exists. Define u\= - Lsxh VI and UI = {u'}. Then Ut = UI and /*(x0) = x\Sx0.
Proof: Suppose JN(UN,x0,X) dénotes the cost with H = X then JN(UL. x0, 0) <
JN(UL, x0, S ) = XQSX0. Thus by the Lebesgue
monotone convergence theorem J„(UL, x0) s
JtïSxo. Also 4 » ï « X o = J « x0, 0) < ƒ*(*„) <
JJJJL x0). Combining these two statements one
has x ï » Ï 0 xos / Ï ( x0) s / „ ( l / : , x0) s x ï S x0. Now
38Ï0-> S, thus /*(x0) = JLClTij x0) = xJSx0. •
The closed loop S y s t e m may not be ms-stable. The next theorem states the conditions for which it is.
Theorem 5.3. Assume that 5 = l i m 33*0
exists. Then R>0, (<I>, Q"2) ms-observable v
Q>0^s4Ls stable, S > 0, S the unique non-negative definite solution of the équation S = 38*S.
Proof: By the same arguments as in Theorem
5.1, S is a solution of the generalized algebraic Riccati équation S = 38» S a n d S a O . Thus S = 38*S = 33i sS = ^L sS + 0 _ + L Î R Ls, S > 0 . Also
by Theorem _ 4.5 R > 0, (<t>, Q'n) ms-observable v Q > 0 (VLs, (Q + Ls-RLs)"2)
ms-observable. Then by Lemma 3.2, MLs stable a n d
S>0. In fact the above arguments hold for a n y
solution S a O of S=38*S. Furthermore by Theorem 4.4
3 8 £ 0 £ 2 ä £ H £ 3 8 £ . t f
= S S*Ls(Q + LTRL) + s4NLSH = 38?s0
+ si^H s 38 £SS + s4ïsH = S + siNLsH. Because 3 8 * 0 - » S and s4Ls is stable we have, taking the limits, 3 8 £ H - > S , Vtf. Suppose S > 0 is any other solution of S = 3B*S. Then S = 3&ÜS
and, taking the limits, S = S. • The statement that MLs is stable is equivalent
to the statement that the closed loop system is ms-stable where L = Ls.
Summarizing loosely we may state that if (<I>, O is ms-stabilizable then the infinite horizon optimal control problem has a solution. If in addition (<!>, Ç>"2) is ms-observable, R>0 or
Q > 0 then the solution is unique and the closed
loop System is ms-stable.
Finally in this section a resuit which relates ms-stabilizability and convergence of the Ric-cati équation (18) is stated.
Corollary 5.1. Assume that R>0, (<P, Q'n) observable or that Q > 0. Then (<I>, O ms-stabilizable O {38*0} converges.
Proof: By Theorem 5.1 (<t>, T)
ms-stabilizable {38*^0} converges. In fact ras-observability, R > 0 or Q > 0 is not needed here. If S = lim 38^0 exists then by Theorem 5.3 R > 0, (<i>, Q"2) ms-observable v Q > 0 4> siLs stable. Thus (4>, T) is ms-stabilizable. •
This corollary emphasizes the important rôle of ms-stabilizability in the solution of the infinite horizon optimal control problem. Loosely speaking it says that under conditions which are almost always met, ms-stabilizability is not only a sufficiënt but also a necessary condition for the existence of the solution of the infinite horizon optimal control problem.
6. C O N D I T I O N S F O R S T A B I L I Z A B I L I T Y
In this section a simple sufficiënt and neces-sary condition, explicit in the system matrices, for S y s t e m s with independent stochastic parameters to have the désirable property of ms-stabilizability is given. This condition also holds for stabilizability in the usual sensé of deterministic S y s t e m s . Furthermore the con-dition is illustrated with some examples. First a spectral radius and two transformations concerning the transformation siL defined by (6) is introduced. The spectral radius p is defined by
p = min p(siL) (23)
L
1. e. p is the minimal spectral radius of s4L achievable through L. The transformation
slt: S" ->5?" is defined by
M*x = siLxx, x e s r (24)
where Lx is defined by
LX = ( rTx r y rTx < p , x G y . (25)
In fact sitf is the same as 38* defined by (18) with Q = R = 6. From (20) s4*X may be written as
jrf*x = <i>Tx*-<i>Txr(rTxr)+rTA:<p, x e s r .
(26) The transformation sii,: is defined by
diiX = , , :. • • • stLlx, x G y \ i > 1. (27) Although si* is nonlinear, the transformation
si±j is linear, i.e. si*, G 3~". The next theorem gives a necessary and sufficient stabilizability condition.
Infinite horizon control of Systems with stochastic parameters 151
Theorem 6.1. (<i>, F) ms-stabilizable <=> p < 1. Proof: Using Theorem 4.4 one has (<t>, T)
ms-stabilizable<=>3L3 p(dL)< 1 « m i n p(dL)< 1.
L
•
This condition would be of less practical value if p could not be determined. However we have the following important resuit.
Theorem 6.2. p = lim 7||'"
Proof: Using Theorem 4.4 one has dil s d¡l 4> lim \\s4il\\<" s lim \\dil\\lli = p(dL)
^ I i m | k ¿ / | r " s p .
Also, using the définition of p, one has p(^,v_.,) a p, i > 1 4> p ( i á * í ) a p\ thus l | | =
IWifll = \stitl*.p(.díi)^P and U m | W / | r " a p. Combining the two results proves the
theorem. • Theorems 6.1 and 6.2 together give a
neces-sary and sufficient condition, which is also ex-plicit in the System matrices and easy to calcú-late for (í>, D to be ms-stabilizable.
A direct c o n s é q u e n c e of Theorems 6.1 and 6.2 is the following interesting result.
and <&h T, replaced by <f>¡,
then from (26) y •y,. Assume y2 ^ 0, (t>2X 4 ¿ X y <l>y2\ x dil y
Assume y^O and replace cj>,y by ¡f>, y then
P 0.
Example 2. Consider the system = 4>¡<t>x¡ + y¡Vu¡
which is the same as syrtem (2) with <i>„ T, replaced by </>¡<t>, y¡T wliere <I>, T are deter-ministic and constant and <t>¡, y¡ scalar stochastic variables, independent in time. Assume T non-singular and y2 ¿ 0 then from (26)
Corollary 6.1. (4>, T) ms-stabilizable O si il -* 6.
Proof: (<I>, O ms-stabilizable O p <
I O I I J Í Í / I H P ' ^ O O J Í ; / - » » . • This result may also be viewed as a
stabil-izability condition in terms of the limiting behaviour of dil.
The importance of p is emphasized by the next corollary.
Corollary 6.2. \\dil\\^p'.
Proof: Follows immediately from Theorem
6.2. • Realizing that from (17) with H = I and Q =
R = 0, JUx«) = xl*s = x¡d^Ix0 or \\x¿f =
xld^lxa, we see that p may also be interpreted as
a measure of the maximal ms-stability of the closed loop system achievable through L . A c -cording to Section 3 all the above results also apply if ms is replaced by m and <J>, F by <P, T. If <t> and T are deterministic and constant, the above results still apply if ms is deleted.
Finally in this section the above stabilizability conditions are illustrated with some examples.
Example 1. Consider the system xM = fax, + y¡u¡ d^X = <t>2<¡>TX<t>
tí
y2tí
y2 <t>r í >Tx r ( rTx r ) TTx * *TrT,rTx r x( rTx r ) TTx r r '<s> = <¿2<t>Tx* - *TrT - ' r x r r ' * y y = fi-&) p=fi Y g y2 ) P2(4>).Assume y¿ 0 and replace <j>, y by <j>, y then p = 0. This example is the case studied by K u and Athans (1977).
Example 3. Consider the system
452 W . L . D E K O N I N G variable, specified by
P {a, = 1} = P {a, = 2} = 0.5.
Then from (26) and Theorem 6.2 we calculate (Stewart, 1973)
p = 1.88 and with a, replaced by â = 1.5
p = 0.
Thus this System is m-stabilizable, but not ras-stabilizable. According to Corollary 6.1, {si^I} should not converge to zéro. That this is indeed the case, is illustrated in Table 1, where the norm of the matrix siil is given for a few values of ;.
In connection with example 3 a few practical remarks concerning the calculation of siil,
\\siil\\ and p are stated. From (26) one sees that
the terms of si^X which dépend on X are ail of the form <S>TX<!>, Wxf or TTXT. Calculation of thèse terms every time X changes might be time consuming especially since there is an expec-tation involved. However, it is sufficient to cal-culate T ® T only once by realizing that e.g.
<PTXr = st"1 [(r 0 *)Tst (X)], where st- 1 means
the inverse of the stack operator st: M1"" - » â?""1.
The st and st 1 opération involves, of course, only
the renumbering of computer memory locations. Furthermore one remarks that st*I G Sf", thus II^IH = p(sîll). Then calculation of the norm is the same as calculation of the spectral radius which is an easy task.
The above remarks also apply to the cal-culation of and ||S8ii/||. For the calcal-culation of p from Theorem 6.2 one may use any matrix norm because of the équivalence of norms on a finite dimensional linear space. Finally one re-marks that often | | ^ i / | | / | | ^ iMJ | | converges faster
to p than \\sijfIf1'. Using this method in Example 3 the value p = 1.88 is reached within 16 itérations in a negligible time on a computer with two 16 bit words floating point arithmetic.
T A B L E 1. D I V E R G I N G RIC-CATl EQUATION K / l l 2 8.1 27.1 3 56.4 4 95.6 5 154.0 6 255.9 7 451.5 8 836.5 9 1586.1 10 3018.3 7. C O N C L U S I O N S
A substantial contribution to the solution of the infinité horizon optimal control problem in the gênerai case of linear discrète time Systems and quadratic criteria, both with independent stochastic parameters has been made.
The properties of stabilizability and ms-observability are introduced for linear discrète time Systems with independent stochastic parameters. ms-Stabilizability appears to be a stronger, and ms-observability a weaker, pro-perty than the usual properties of stabilizability and observability for deterministic Systems.
It has been shown that, loosely speaking, if the System is ms-stabilizable then the infinité horizon optimal control problem has a solution. If in addition the problem is ms-observable then the solution is unique and the closed loop Sys-tem is stable in the mean square sensé. Fur-thermore, if the problem is ms-observable then ms-stabilizability is not only a sufficient but also a necessary condition for the existence of the solution.
Finally a simple sufficient and necessary condition, explicit in the System matrices, for Systems with independent stochastic parameters to be ms-stabilizable has been given. This con-dition also holds for stabilizability in the usual sensé of deterministic Systems. The condition has been illustrated with some examples.
With the aid of monotonie transformations on finite dimensional Banach spaces with a partial ordering we have been able to generalize some concepts of deterministic Systems to Systems with stochastic parameters. This will enhance our insight in Systems where the uncertainties are represented by stochastic parameters in-stead of additive noise.
R E F E R E N C E S
A o k i , M . (1967). Optimization of Stochastic Systems. Academie Press, New York.
Äthans, M . , R. K u and S. B . Gershwin (1977). The un-certainty threshold principle: some fundamental limita-tions of optimal décision making under dynamic un-certainty. IEEE Trans. Aut. Control, AC-22, 491. Barnett, S. (1971). Matrices in Control Theory. V a n
Nos-trand and Reinhold, London.
Bellman, R. (1970). Introduction to Matrix Analysis. M c G r a w - H i l l , New Y o r k .
Burrill, C . W . (1972). Measure, Integration and Probabllity. M c G r a w - H i l l , New York.
Connors, M . M . (1967). Controllability of discrete, linear, random dynamical Systems. J. Siam Control, 5, 183. Drenick, R. F . and L . Shaw (1964). Optimal control of linear
plants with random parameters. IEEE Trans Aut. Control, A C - 9 , 236.
Harris, S. E . (1978). Stochastic controllability of linear dis-crete Systems with multiplicative noise. Int. J. Control, 27, 213.
Kalman, R. E . (1961). Control of randomly varying linear dynamical Systems. Proc. Symp. Appl. Math., 13, 287.
Infinite horizon control of Systems with stochastic parameters 453 Katayama, T. (1976). On the matrix Riecati equation for
linear Systems with a random gain. IEEE Trans Aut.
Control, AC-21, 770.
Krasnosel'skü, M . A . (1%4). Positive Solutions of Operator
Equations. Noordhoff, Groningen.
Kreyszig, E . (1978). Introductory Functional Analysis with
Applications. John Wiley, New York.
K u , R. T. and M . Äthans (1977). Further results on the uncertainty threshold principle. IEEE Trans Aut. Control, AC-22, 866.
Kwakernaak, H . and R. Sivan (1972). Linear Optimal
Con-trol Systems. John Wiley, New York.
Stewart, G. W. (1973). lntroduction to Matrix Computations. Academic Press, New York.
Detectability of linear discrete-time Systems with
stochastic parameters
W . L . D E K O N I N G t
The paper introduces the concept of moan square detectability and relates this to the recently introduced concept of mean square observability. It is shown that under appropriate mean square detectability and stabilizability conditions the infinité-horizon optimal control problem for the gênerai case of linear discrète time Systems and quadratic criteria, both with stochastic parameters which are statistically inde-pendent of time, has a unique solution when the control System is mean square stable. A simple necessary and sufficient condition, explicit in the System matrices, is given to d é t e r m i n e if a System is mean square d é t e c t a b l e . This condition also holds for deterministic Systems to be d é t e c t a b l e in the usual s e n s é . The mean square detecta-bility property coincides with the usual one if the parameters are deterministic. 1. Introduction
I t is well k n o w n ( K w a k e r n a a k and Si van 1972) that under appropriate stabilizability and detectability conditions the usual i n f i n i t é - h o r i z o n L Q problem for a deterministic System has a unique solution when the control System is stable. I n the practically more i m p o r t a n t case of Systems, and perhaps criteria, w i t h stochastic parameters the situation is more complicated. The question therefore arises as to the conditions on the statistical properties of the System and criteria matrices for w h i c h the infinité-horizon problem has a solution and the control system is stable.
W e shall be concerned w i t h linear discrete-time Systems and quadratic criteria b o t h h a v i n g stochastic parameters w h i c h are statistically independent of time. F o r succinctness t h è s e w i l l be referred to as independent stochastic parameters.
The i n f i n i t é - h o r i z o n o p t i m a l control of linear discrete-time Systems w i t h independent stochastic parameters has been studied b y K a l m a n (1961), Drenick and Shaw (1964), K a t a y a m a (1976), A t h a n s et al. (1977) and K u a n d A t h a n s (1977). T h e y ail found i n spécial cases necessary and sufficient conditions for the existence of the solution of the i n f i n i t é - h o r i z o n problem. D e K o n i n g (1982) solved the i n f i n i t é - h o r i z o n o p t i m a l control problem for the g ê n e r a i case of linear discrete-time Systems and quadratic criteria b o t h w i t h independent stochastic parameters b y generalization of the usual notions of stabilizability and observability for deterministic Systems, called mean square stabilizability and mean square observability. I t appears that under appro-priate mean square s t a b i l i z a b i l i t y and observability conditions the o p t i m a l control problem has a unique solution when the control system is mean square stable.
I n this paper we introduce a generalization of the usual n o t i o n of detecta-b i l i t y for deterministic Systems, called mean square detectadetecta-bility. I t is shown t h a t for the solution of the above problem the condition of mean square
Received 19 January 1983.
f Department of Applied Physics, Delft University of Technology, Prins Bernhardlaan 6, 2628 B W Delft, The Netherlands.
