A simulation of integral and derivative of the solution of a stochastic

ANNALES

POLONICI MATHEMATICI LVII.1 (1992)

A simulation of integral and derivative of the solution of a stochastic

integral equation

by Nguyen Quy Hy and Nguyen Thi Minh (Hanoi)

Abstract. A stochastic integral equation corresponding to a probability space (Ω, Σ

, P

) is considered. This equation plays the role of a dynamical system in many problems of stochastic control with the control variable u(·) : R

→ R

. One constructs stochastic processes η

(t), η

(t) connected with a Markov chain and with the space (Ω, Σ

, P

). The expected values of η

(t) (i = 1, 2) are respectively the expected value of an integral representation of a solution x(t) of the equation and that of its derivative x

(t).

1. Introduction. Given a probability space (Ω, Σ

, P

). Let L

(Ω) be the Hilbert space of all real random variables defined on (Ω, Σ

, P

) which have finite second moment:

(1.1) L

(Ω) = n

ξ : Ω → R

 kξk

:= h R

Ω

ξ

(ω)P

(dω) i

= [E

{ξ

(ω)}]

< ∞ o

. We consider the stochastic equation

(1.2) x(t) =

b

R

a

K(t, τ )x(τ ) µ(dτ ) + g(t, u(·)) (t ∈ ha, bi (mod µ)) where ha, bi is a closed or open interval, −∞ ≤ a < b < ∞, µ is Lebesgue measure on R

, u(t) ∈ R

, x(t) = (x

(t), . . . , x

(t))

, x

(t) ∈ L

(ha, bi).

We suppose that the given functions in (1.2) satisfy the following condi- tions:

(A) The vector-valued function u(t) = (u

(t), . . . , u

(t))

: ha, bi → (u, u) ⊂ R

is deterministic (where (u, u) = {(u

, . . . , u

) : u

<

1991 Mathematics Subject Classification: Primary 60H20.

2 N g u y e n Q u y H y and N g u y e n T h i M i n h

u

< u

, i = 1, . . . , m}). The function g(t) = (g

(t), . . . , g

(t))

:=

g(t, u(·); ω) is a Hilbert valued n-variate process with parameter t ∈ ha, bi, integrable on (a, b) (see [8]).

(B) The matrix-valued function K(t, τ ; ω) := (K

(t, τ ; ω))

is a Hil- bert valued (n × n)-variate process with parameters (t, τ ) ∈ ha, bi × ha, bi.

Under some assumptions, there exists a unique solution of (1.2) which depends on u(·) : x(t) = x(t, u(·); ω) and is a Hilbert valued n-variate process with parameter t ∈ ha, bi, i.e. (see [8]) x

(t) = x

(t, u(·)) ∈ L

(Ω). In many general problems of stochastic control, we deal with the state equation of the form (1.2) (see [15], [5], [11], [12]) with K(t, τ ) = 0 for τ > t ≥ a; x(t) is the state variable, u(t) is the control variable.

Using gradient methods to solve the corresponding stochastic control problems, we must determine the random gradient Φ

(x, u, t; ω) of some average cost E{ R

a

Φ dt}. This problem reduces to determining the expected value:

x(t) := E

{x(t, u(·); ω)} , (1.3)

x

(t) := E

{x

(t, u(·); ω)} , (1.4)

where the components of the matrix x

:

∂x

(t, u(·); ω)

∂u

= ∂x

(t, u; ω)

∂u

are the mean quadratic derivatives (m.q.d.) of the stochastic process x

(t, u

, . . . , u

; ω) with respect to the parameter u

(see [8] or (3.1b)). Fur- ther, we have to determine the following vector and matrix:

hϕ

, xi := E

n R

a

ϕ

(t; ω)x(t, u(·); ω) µ(dt) o

, (1.5)

hϕ

, x

i := E

n R

a

ϕ

(t; ω)x

(t, u(·); ω) µ(dt) o

. (1.6)

Here ϕ

(i = 1, 2) satisfy the following condition:

(C

) ϕ

(t; ω) is a Hilbert valued (n

×n)-variate process with components having second moment integrable on ha, bi.

When n

= n and ϕ

(t; ω) is the unit matrix of order n, problem (1.5)

reduces to determining the expected value of the integral of the solution of

(1.2) on ha, bi. We can also investigate the similar problem for the integral

of the derivatives of the solution.

Simulation of integral and derivative 3

In particular, when n = m = 1, u(t) = t, and K(t, τ ) and g(t, u(·)) = g(u(t)) = g(t) are deterministic, (1.2) takes the form of a Fredholm equation of the second type in L

(a, b). Then an unbiased estimator (u.e.) and an asymptotic u.e. of (1.3), (1.5) are obtained in [14], [10], [9].

When g(t) = E

{η(t; ω)} and K(t, τ ) = E

{χ(t, τ ; ω)}, analogous esti- mators are constructed basing on a realization of the processes η

, χ(t, τ ) (see [7], [6], [4], [1]). An asymptotic u.e. and an u.e. of (1.4), (1.5) are also constructed for χ(t, τ ) ≥ 0 (see [3]).

In this paper, we provide an u.e. of (1.3)–(1.6) for the stochastic integral equation (1.2).

By the above method we can apply the Monte Carlo method to the numerical calculation of the quantities (1.3)–(1.6).

2. A random model connected with an integral transformation of a solution. Let L

= L

(Ω × (a, b)) be the class of Hilbert valued processes with second moment integrable on ha, bi:

(2.1) L

= n

f : ha, bi → L

(Ω)    kf k

1×1

=

b

R

a

E

{f

(t)} µ(dt) < ∞ o . Then L

is a Hilbert space (see [2]).

Let L

= L

(Ω ×(a, b)) be the class of Hilbert valued (n×s)-variate processes with components having second moment integrable on (a, b):

(2.2) L

= n

F = (F

)

 F

: ha, bi → L

(Ω) ,

b

R

a

kF

(t)k

µ(dt) =

b

R

a

R

Ω

F

(t; ω)P

(dω) µ(dt) < ∞ o

. We can represent L

in the form L

= L

⊕ . . . ⊕ L

(n × s sum- mands). Hence L

is also a Hilbert space (see [2]) with the norm

(2.2

) kF k

n×s

=  X

i=1 s

X

j=1

kF

k



(∀F ∈ L

) . Let K and K

be the following integral operators:

(2.3) [Kf ](t) =

b

R

a

K(t, τ )f (τ )µ(dτ ) (f ∈ L

) ,

(2.4) [K

f ](t) =

b

R

a

K

(t, τ )f (τ )µ(dτ ) (f ∈ L

) ,

where for each matrix H = (H

)

we write H

= (|H

)

. We

suppose that: