Dynamie Programming with Returns in Random Variables Spaces

A C T A U N I V E R S I T A T I S L O D Z I E N S I S

FO LIA O EC O N O M IC A 175, 2004

S e b a s t i a n S i t a r z *

D Y N A M IC P R O G R A M M IN G W IT H R E T U R N S

IN R A N D O M V A R IA B LES S P A C E S

Лlistrad This paper presents a m odel o f dynam ic, discrete decision-m aking problem (finite number o f periods, states and decision variables). Described process has returns in random variables spaces equipped with partial order. The model can be applied for many multi-stage, multi-criteria decision m aking problem s. There are a lot o f order relations to com pare random variables. Properties o f those structures let us apply Bellman’s Principle o f dynamic programming. The result o f using this procedure is obtainm ent o f a whole set o f optim al values (in the sense o f order relation). For illustration, there is presented a numerical exam ple.

Key words: dynam ic program ming, partially ordered set, stochastic dom inance.

L IN T R O D U C T IO N

T h e p ap e r p resents an o p tim al m odel involved into a shap e o f dyn am ic

program m ing. T h is th eo ry was intro duced by R. Bellm an (a m eth o d o f

solving such tasks is nam ed B ellm an’s principle). N ext T . A. B r o w n and

R. E. S t r a u c h (1965) generalized B ellm an’s principle to a class o f

m u lti-criteria dynam ic p ro g ram m in g with a lattical o rd er. T h en , th e use o f

optim ality principle w as the interest o f L. M i t t e n (1974) w ho considered

preferences relatio n , M . I. H e n i g (1985) - w ho developed the th eo ry o f

infinite d ynam ic process with values o f criteria fu n ctio n in a p artially

ordered set. O thers w ho to o k interest in th e use o f m ulticriterial m eth o d s

in dynam ic p ro g ram m in g have been: T . T r z a s k a l i k (1998), D . L i and

Y. Y. H a im e s (1989).

In the m ean tim e som e theories o f com parin g ra n d o m variables has

developed as well 'Г. R o l s k i (1976), M . S h a k e d a nd J. G. S h a n t -

h i k u m a r (1993). It enabled us to use such stru ctu re s in o u r dyn am ic

m odel, w hich is th e essence o f th e paper. In the exam ple, th ere is show n

* M A , In stitu te o f M a th em a tics, U n iversity o f S ilesia, K a to w ice , ssitarz@ - ux2.m ath.us.edu.pl.

a com bination o f fields, nam ely, dynam ic program m ing and rand om variables

with stochastic dom inance.

2. D Y N A M IC M O D E L

W e consider m u ltistag e dyn am ic process with finite n u m b er o f periods,

states and decision variables. T o describe the process we will use the

follow ing n o tatio n :

T - the n u m b er o f periods,

S, - the set o f all feasible state variables at th e beginning o f period

t e { 1, ..., T ) ,

D,(st) - the set o f all feasible decision variables for period t and state

s , e S „ we assum e th a t all these sets are finite,

P - den o tes the process, w here all sets: T, S,, Dt(s,) are identified, on

the base o f these term s we define:

r, = (sf, s(+ i) - th e period realization, s , e S , an d st+1 e D , ( s t),

R , - the set o f all period realizations in period £,

(s„ ..., s r +i ) - the partial realization in period t an d swe S w, and

S w + i ^ D J s J for w e ( t ,

T),

R,(s,) - th e set o f all p a rtia l realizatio n s w hich begin a t s ta te s„

R,(St) = R t(st) : s , e S t - the set o f all p artia l realizations which begin at the

beginning o f period t,

R = R i ( S j) - the set o f all process realizations,

We consider the follow ing stru ctu re, functions and o p e ra to rs to describe

m ulti-period criteria fun ctio n o f process realization.

(W, < , °) - the stru c tu re in which ( W., < ) is the partially ord ered set, and

(W, °) is sem igroup satisfies follow ing co nd ition

a ^ b = > ( a ° c ^ b o c an d c ° a ^ c ° b ) \ a, b, c e W (m o n o to n icity co n d itio n )

(

1

)

F o r each finite subset A c W we define

т а х ( Л ) s { a * e A : ~ Э 0бЛа * < а and

а* Ф a)

(2)

Values o f the crite ria fun ctio n are given by the stru ctu re ( W, ^ , °)

f t : R t —* W - the period criteria functions with re tu rn s in W.

F, : R, (St) —* W - the fu nctions defined in the follow ing way

t = T , .... 1

(3)

F = F l - the m ulti-p erio d criteria function,

(P , F) - d enotes discrete dynam ic decision process. It is given, if there

arc discrete d ynam ic process P and m ulti-period criteria fu n ctio n F d e­

fined.

R ealization d * e D is said to be efficient, if

F( d * )e m ax F(D )

(4)

Theorem 1. Let (P , F) be decision dynam ic process.

0

)

F o r all t = T — 1,

1 and all у ,е У , holds

m ax {F,(R,(s,)} = m ax

s( + 1) ° m a x F, + 1( R t +l (s, + l ) ) : s f + l e D f(s,)}

(5)

(ii)

m ax {F( R) } = m a x jm a x F 1( R l (si ) ) : s i e S l }

(6)

3. P R O C E D U R E

W c now p re sen t an a lg o rith m fo r d e te rm in a tio n o f th e set o f all

m axim al re tu rn s o f th e process, w hich is based on T h . 1. T his p rocedu re

is stated as follows:

step 1

C alculate the set: m a x ^ ^ K ^ ) } fo r all states sT e S T.

step ( T + 1 - t ) , for t = T - 3, T — 4,

1.

C alculate th e set: m a x { F f(R f(st)) using T h. 1 (i).

step T + 1

C alculate the set: m ax{F (R )} using T h. 1 (ii).

4. E X A M PLE

T h e alg o rith m presented in the th ird section is now applied to solve

dynam ic problem . W e use d ifferent o rders to co m p are values o f th e criteria

function i.e. ran d o m variables. T h e n o tatio n below agrees with sym bols

previously used.

T h e process (P , F ) is defined as follows:

We consider a process, which consists o f 3 periods [ T = 3], in which:

S, = {0,1}, for t = 1, 2, 3, 4; D,(0) = D ,(l) = {0, 1}, for t = 1, 2, 3 (7)

T he term s connected w ith the criteria fu nction arc defined as follows. Set

W is described as the set o f discrete random variables

W =

Pi, P2, -, P„)' neN, pn>0, p^O,

1 = 0

where

pt

den o tes p ro b ab ility o f num ber i.

O p e rato r p is defined as a sum o f ran d o m variables.

T h e values o f the crite ria function are presented in the Fig. 1.

5. T H E O R D E R S U S E D IN T H E EX A M PL E

T o co m p are such values o f the criteria function we use k no w n orders

generated by:

1) first o rd e r sto ch astic d om in ance F S D ,

2) second o rd e r sto ch astic dom in an ce SSD,

3) second o rd e r inverse stochastic d om inan ce SISD ,

4) m ean-variance m odel (as two-criteria: m axim izing m ean and m inim izing

variance).

T h o se classical defin ition can be found am ong o th ers in M . S h a k e d

a nd J. G. S h a n t h i k u m a r (1993) - stochastic orders; and m ean -v arian ce

m odel in H . M . M a r k o v i t z (1989). T h e relations used in this exam ple

arc n o t antisym m etric, w hich is one o f the condition s o f p artia l o rd e r, b u t

one can c o n sid e r th e eq u iv ale n ce re la tio n (x < 'у о x < у an d y < x )

G. B i r k h o f f (1973). T h e m o n o to n icity co n d itio n s o f these stru ctu re s are

show n in M . S h a k c d and J. G. S h a n t h i k u m a r (1993). M o re o v er the

rest o f the co n d itio n s w hich are needed to hold the T h . 1 are easy to check.

6. R ESULTS

Below, th ere arc results o f using algorithm presented. T h e tables show

m axim al values o b tain ed in each step.

T a b l e 1

stochastic dom inance (SSD ). The values connected with SS D case are bold

I m ax{F ,(R ,(0)) m a x {F ,(R ,(l)) 3 (0,1) (.2, .3, .5), (0, .6, .4) 2 (0, .2, .3, .5), (0, 0, .6, .4) (0, 0, 1), (.1, .25, .4, .25), (0, .3, .5, .2) 1 (0, 0, .2, .4, .4), (.02, .09, .22, .31, .26, .1), (0, .06, .22, .36, .28, .08) (0,

0

0

The com putation in the case o f second order inverse stochastic dom inance (SISD )

t m ax{F ,(R ,(0)) m ax {F ,(R ,(l)) 3 (0, 1) (.2, .3, .5), (0, .6, .4) 2 (0, .2, .3, .5), (0, 0, .6, .4) (0, 0, 1), (.1, .25, .4, .25), (0, .3, .5, .2) 1 (0, 0, .2, .4, .4) (.02, .09, .22, .31, .26, .1) (0, .06, .22, .36, .28, .08) (0, 0, .24, .52, .24), (0, .08, .24, .38, .3) max F (R ) (0, 0, .2, .4, .4), (.02, .09, .22, .31, .26, .1), (0, .06, .22, .36, .28, .08)

The com putation in the case o f m ean-variance model. There arc values o f the mean and variance, instead o f elem ents o f W, in the follow ing form: (mean, variance)

t (mean, variance) o f nrnfF ^R ^ O )) (mean, variance) o f m a x {F ,(R ,(l))

3 (1, 0) (1.4, .24)

2 (1.8, .16), (2.4, .24) (2, 0)

1 (3.2, .56), (2.9, .49), (2.3, .41) (3, 0) (mean, variance) o f max F (R )

(3.2, .56), (3, 0)

max F (R ) = (0, 0, .2, .4, .4), (0, 0, 0, 1)

REFEREN CES

B i r k h o f f G . (1973), L a ttice Theory, American Mathematical Society, Colloquium Publications, 25.

B r o w n T. A. , S t r a u c h R. E. (1965), D ynam ic Program m ing in M ultiplicative L attices,

J. M ath. A nalysis and Applications, 12, 2, 364-370.

F u c h s L. (1963), P artially O rdered Algebraic System s, “International Series o f M onographs on Pure and A pplied M athem atics” , 28.

H e n i g M. I. (1985). The Principle o f O ptim ality in D ynam ic P rogram m ing with R eturns in

P artially O rdered S ets, M ath, o f Oper. Res., 10, 3, 462-470.

L i D ., H a i m e s Y. Y . (1989), M ultiobjective D ynam ic Program m ing: The S ta te o f the A rt, “Control Theory and A dvanced T ech nology” , 5, 4, 471-483.

M a r k o v i t z H. M. (1989), M ean-V ariance A nalysis in Portfolio Choice a n d C apital M a rkets, Basic Blackwell, O xford-C am bridge.

M i t t e n L. (1974), Preference Order D ynamic Programming, “Management Science", 21, 1, 43-46. R o l s k i T. (1976), O rder R elations in the S et o f Probability D istributions and their Applications

in the Q ueneining Theory, “ D issertation M athem aticae” , 132.

S h a k e d M. , S h a n t h i k u m a r J. G. (1993), Stochastic Orders a n d their Applications, Academ ic Press, Harcourt Brace & C o., Boston.

S z e k l i R. (1995), S to ch a stic Ordering a n d Dependency in A pplied Probability, Springer Verlag, N ew York.

T r z a s k a l i k T . (1994), M u ltip ly C riteria Discrete D ynam ic Program m ing, “ M athem atics T od ay” , XII-A, 173-199.

T r z a s k a l i k T. (1998), M ultiobjective Analysis in D ynam ic E nvironm ent, T he A cadem y o f Econom ics, K atow ice.

T r z a s k a l i k T. , S i t a r z S. (2001), D ynam ic Discrete P rogram m ing with P artially O rdered

Criteria S e t, [in:] T . T r z a s k a l i k , J. M i c h n i k (eds.), M ultiobjective a n d Goal P ro­ gram m ing, Springer Verlag, 186-195.

W h i t e m o r e G. A. , F i n d l a y M. C. (eds.) (1978), Stochastic D om inance: A n Approach to

Sebastian S ita rz

Z M IE N N E L O S O W E W D Y SK R ETN Y M PR O G R A M O W A N IU D Y N A M IC Z N Y M

W artykule opisano dyskretny model program owania dynam icznego z w artościam i funkcji kryterium z przestrzeni zmiennych losow ych wyposażonej w częściow y porządek. Opisany proces dynam iczny ma charakter deterministyczny. Porównując zmienne losow e stosow ane są różne rodzaje relacji porządkujących. W łasności struktur zm iennych losow ych pozw alają stosow ać uogólnioną m etodę program owania dynam icznego - tzw. zasadę Bellmana. Efektem tej procedury jest uzyskanie pełnego zbioru wartości optymalnych (w sensie relacji częściow ego porządku). A nalogicznie, jak w program owaniu wielokryterialnym, tak i tu rozwiązaniem problemu optym alizacyjnego m oże być duży zbiór wartości optym alnych. Przedstawione są metody zawężające ten zbiór, wykorzystujące dynam iczną postać zadania oraz własności zmiennych losow ych.