A TWO-DISORDER DETECTION PROBLEM

K. S Z A J O W S K I (Wroc law)

A TWO-DISORDER DETECTION PROBLEM

Abstract. Suppose that the process X = {X n , n ∈ N} is observed sequen- tially. There are two random moments of time θ ₁ and θ ₂ , independent of X, and X is a Markov process given θ 1 and θ 2 . The transition probabilities of X change for the first time at time θ 1 and for the second time at time θ 2 . Our objective is to find a strategy which immediately detects the distribution changes with maximal probability based on observation of X. The corre- sponding problem of double optimal stopping is constructed. The optimal strategy is found and the corresponding maximal probability is calculated.

1. Introduction. Suppose that a process X = {X _n , n ∈ N} (N = {0, 1, 2, . . .}) is observed sequentially. The process is obtained from three Markov processes by switches between them at two random moments of time, θ 1 and θ 2 . Our objective is to detect immediately these moments based on observation of X.

This type of problem arises in quality control. An automaton which produces some details changes its parameters. This causes the details to change their quality. Production can be divided into three grades. Assuming that at the beginning of the production process the quality is highest, from some time θ 1 on the products should be classified to a lower grade, and beginning with θ 2 to the lowest grade. We want to detect the moments of these changes.

Shiryaev (1978) considered the disorder problem for independent random variables with one disorder where the mean distance between disorder time and the moment of its detection was minimized. The probability maximiz- ing approach to the problem was used by Bojdecki (1979), and the stopping time which is in a given neighbourhood of the moment of disorder with

1991 Mathematics Subject Classification: Primary 60G40; Secondary 62L15.

Key words and phrases : disorder problem; sequential detection; multiple optimal stopping.

[231]

maximal probability was found. The problem with two disorders was con- sidered by Yoshida (1983) and Szajowski (1992). Yoshida (1983) solved the problem of optimal stopping for the observation of a Markov process X so as to maximize the probability that the distance between θ _i , i = 1, 2, and the moment of disorder will not exceed a given number (for each disorder independently). He constructed a strategy which stops the process between the first and the second disorder with maximal probability. References to other papers treating variations of the disorder problem can be found in Szajowski (1992).

In the present paper the probability maximizing approach to optimal stopping developed by Bojdecki (1979) is extended to solve a double stopping problem (see Haggstrom (1967), Nikolaev (1979)) arising in the quickest detection of double disorders. In Section 2 the problem is formulated in a rigorous manner. Section 3 contains the reduction of the problem to an optimal stopping problem for a doubly indexed stochastic sequence. The main result is given in Section 4.

2. The double disorder detection problem. Let X = {X n , n ∈ N}, defined on (Ω, F, P ), be a potentially observable sequence of r.v.’s with values in (E, B), where E is a subset of the real line. Assume that the epochs of distributional changes are N-valued F-measurable r.v. θ 1 and θ 2 , independent of X and having the distribution

(1) P (θ 1 = j) = p ^j−1 ₁ q 1 , P (θ 2 = k | θ 1 = j) = p ^k−j−1 ₂ q 2 , where j = 1, 2, . . . , k = j + 1, j + 2, . . . and p _i + q _i = 1, i = 1, 2.

Suppose that on (Ω, F, P ) Markov processes X ⁱ = {(X _n ⁱ , F _n , P _x ⁱ )}, i = 1, 2, 3, are defined and we have

(2) X _n =







X _n ¹ if n < θ ₁ , X _n ² if θ 1 ≤ n < θ 2 , X _n ³ if n ≥ θ 2 .

The measures P _x ⁱ , i = 1, 2, 3, are absolutely continuous with respect to some fixed measure P x and satisfy the following relations: P _x ⁱ (dy) = f _x ⁱ (y)P (x, dy), where f _x ⁱ (·) 6= f _x ^j (·), i 6= j and f _x ⁱ⁺¹ (y)/f _x ⁱ (y) < ∞, i = 1, 2, for every x, y ∈ E. The distribution of θ _i , i = 1, 2, is given by (1) and the measures P _x ⁱ , i = 1, 2, 3, x ∈ E, are known. We observe the process (X n , F n , P x ), n = 0, 1, 2, . . . , x ∈ E, which is a Markov process given θ ₁ and θ ₂ , defined by (2) with F _n = σ(X 0 , X 1 , . . . , X _n ). On the basis of the distribution of θ 1 , θ 2 and measures P _x ⁱ , i = 1, 2, 3, x ∈ E, we calculate the finite-dimensional distributions of the observed process.

Let S denote the set of all stopping times with respect to the filtration

(F _n ), n = 0, 1, . . . , and T = {(τ, σ) : τ < σ, τ, σ ∈ S}. Let us determine a

pair of stopping times (τ ^∗ , σ ^∗ ) ∈ T such that for every x ∈ E, (3) P x (τ ^∗ < σ ^∗ < ∞, |θ 1 − τ ^∗ | ≤ d 1 , |θ 2 − σ ^∗ | ≤ d 2 )

= sup

(τ,σ)∈T

P _x (τ < σ < ∞, |θ ₁ − τ | ≤ d ₁ , |θ ₂ − σ| ≤ d ₂ ).

This problem will be denoted by D _{d 1 d 2} .

3. Reduction of the double “disorder problem” to double opti- mal stopping of a Markov process. A compound stopping variable is a pair (τ, σ) of stopping times such that τ < σ a.e. Define T _m = {(τ, σ) ∈ T : τ ≥ m}, T _mn = {(τ, σ) ∈ T : τ = m, σ ≥ n} and S _m = {τ ∈ S : τ ≥ m}.

Set F mn = F n , m, n ∈ N, m ≤ n. We define a two-parameter stochastic sequence ξ(x) = {ξ _mn (x), m, n ∈ N, m < n, x ∈ E} by

ξ _mn (x) = P _x (|θ 1 − τ | ≤ d 1 , |θ 2 − σ| ≤ d 2 | F _mn ).

For every m, n ∈ N with m < n, we can consider the optimal stopping problem of ξ(x) on T _mn . A compound stopping variable (τ ^∗ , σ ^∗ ) is said to be optimal in T m (or T mn ) if E x ξ τ ^∗ σ ^∗ = sup _{(τ,σ)∈T m} E x ξ τ σ (or E x ξ τ ^∗ σ ^∗ = sup _{(τ,σ)∈T mn} E _x ξ _{τ σ} ). Define

(4) η _mn (x) = ess sup

(τ,σ)∈T mn

E(ξ _{τ σ} | F _mn ), (5) η m = E x (η m,m+1 | F m ).

If we put ξ _m∞ = 0, then η _mn = ess sup

(τ,σ)∈T mn

P _x (|θ 1 − m| ≤ d 1 , |θ 2 − n| ≤ d 2 | F _mn ).

From the theory of optimal stopping for double indexed processes (cf. Hag- gstrom (1967), Nikolaev (1981)) the sequence η _mn satisfies

η _mn = max{ξ _mn , E(η m,n+1 | F _mn )}.

Moreover, if σ ^∗ _m = inf{n ≥ m : η mn = ξ mn }, then (m, σ _m ^∗ ) is optimal in T mn

and η _mn = E _x (ξ _mσ ∗

m | F _mn ) a.e.

Lemma 1. The stopping time σ ^∗ _m is optimal for every stopping problem (4).

P r o o f. It suffices to prove lim _n→∞ ξ _mn = 0 (Lemma 4.10 of Chow, Robbins & Siegmund (1971), cf. also Bojdecki (1979), Bojdecki (1982)).

For m, n, k ∈ N with n ≥ k > m and every x ∈ E we have E _x (I _{{|θ 1 −m|≤d 1 ,|θ 2 −n|≤d 2 }} | F _mn ) = ξ _mn (x)

≤ E x (sup

j≥k

I _{|θ

1 −m|≤d ₁ ,|θ ₂ −j|≤d ₂ } | F n ),

where I _A is the characteristic function of the set A. By Levy’s theorem, lim sup

n→∞ ξ mn (x) ≤ E x (sup

j≥k

I _{|θ

1 −m|≤d ₁ ,|θ ₂ −j|≤d ₂ } | F n∞ ), where F ∞ = F n∞ = σ( S ∞

n=1 F _n ).

We have lim k→∞ sup _j≥k I _{|θ

1 −m|≤d ₁ ,|θ ₂ −j|≤d ₂ } = 0 a.e. and by the dom- inated convergence theorem,

k→∞ lim E _x (sup

j≥k

I _{|θ

1 −m|≤d ₁ ,|θ ₂ −j|≤d ₂ } | F ∞ ) = 0.

As the next step the optimal stopping problem for η m should be solved.

Define

(6) V m = ess sup

τ∈S m

E x (η τ | F m ).

Then V _m = max{η _m , E _x (V _m+1 | F _m )} a.e. and we define τ _n ^∗ = inf{k ≥ n : V _k = η _k }.

Lemma 2. The strategy τ ₀ ^∗ is the optimal strategy of the first stop.

P r o o f. To show that τ ₀ ^∗ is the optimal first stop strategy we prove that P _x (τ ₀ ^∗ < ∞) = 1. We argue in the usual manner, i.e. we show lim m→∞ η m (x)

= 0.

We have η _m = E _x (ξ _mσ ∗

m | F _m ) = E _x (E _x (I _{{|θ 1 −m|≤d 1 ,|θ 2 −σ} ∗

m |≤d ₂ } | F _mσ ∗

m ) | F _m )

= E _x (I _{{|θ 1 −m|≤d 1 ,|θ 2 −σ} ∗

m |≤d ₂ } | F _m )

≤ E x (sup

j≥k

I _{|θ

1 −j|≤d ₁ ,|θ ₂ −σ _j ^∗ |≤d ₂ } | F m ).

Similarly to the proof of Lemma 1 we have lim sup

m→∞ η m (x) ≤ E x (sup

j≥k

I _{|θ

1 −j|≤d 1 ,|θ 2 −σ ^∗ _j |≤d 2 } | F ∞ ).

Since

k→∞ lim sup

j≥k

I _{|θ

1 −k|≤d ₁ ,|θ ₂ −σ ^∗ _j |≤d ₂ } ≤ lim sup

k→∞

I _{|θ

1 −k|≤d ₁ } = 0, it follows that

m→∞ lim η m (x) ≤ lim

k→∞ E x (sup

j≥k

I _{|θ

1 −j|≤d ₁ ,|θ ₂ −σ _j ^∗ |≤d ₂ } | F ∞ ) = 0.

Lemmas 1 and 2 describe the method of solving the “disorder problem”

formulated in Section 2.

4. Immediate detection of the first and second disorder. For the

sake of simplicity we restrict ourselves to the case d 1 = d 2 = 0. It will be

easily seen how to generalize the solution to D _{d 1 d 2} for d 1 > 0 or d 2 > 0.

First we construct multi-dimensional Markov chains such that ξ _mn and η _m are the functions of their states. Set (cf. Yoshida (1983), Szajowski (1992))

Π _n ¹ (x) = P _x (θ ₁ > n | F _n ), Π _n ² (x) = P _x (θ ₂ > n | F _n ), Π mn (x) = P x (θ 1 = m, θ 2 > n | F mn ) for m, n = 1, 2, . . . , m < n,

H(t, u, α, β) = αp 1 f _t ¹ (u) + [p 2 (β − α) + q 1 α]f _t ² (u) + [1 − β + q 2 (β − α)]f _t ³ (u), Π ¹ (t, u, α, β) = p ₁ αf _t ¹ (u)(H(t, u, α, β)) ⁻¹ ,

Π ² (t, u, α, β) = {p 1 αf _t ¹ (u) + [αq 1 + (β − α)p 2 ]f _t ² (u)}(H(t, u, α, β)) ⁻¹ , Π(t, u, α, β, γ) = p 2 γf _t ² (u)(H(t, u, α, β)) ⁻¹ .

The following auxiliary results will be needed in the proof of the main the- orem.

Lemma 3. For each x ∈ E and m, n = 1, 2, . . . with m < n, and each Borel function u : R → R,

Π _n+1 ¹ (x) = Π ¹ (X _n , X n+1 , Π _n ¹ (x), Π _n ² (x)), (7)

Π _n+1 ² (x) = Π ² (X n , X n+1 , Π _n ¹ (x), Π _n ² (x)), (8)

Π m,n+1 (x) = Π(X _n , X n+1 , Π _n ¹ (x), Π _n ² (x), Π _mn (x)), (9)

with the boundary condition Π ₀ ¹ (x) = Π ₀ ² (x) = 0, Π mm (x) = q 1 f _X ²

m−1 (X m ) p 1 f _X ¹

m−1 (X m ) Π _n ¹ (x) and

(10) E _x (u(X n+1 ) | F _n ) =

\

E

u(y)H(X _n , y, Π _n ¹ (x), Π _n ² (x)) P _{X n} (dy).

P r o o f. (7), (8) and (10) are proved in Yoshida (1983) and Szajowski (1992). The formula (9) follows from the Bayes formula:

P x (θ 1 = j, θ 2 = k | F n )

=



 

 

 

 

 

 

 

 

P x (θ 1 = j, θ 2 = k)p ⁿ ₁ Q n

s=1 f _x ¹ _s−1 (x s )

×(S n (x 0 , x 1 , . . . , x n )) ⁻¹ if j > n, P _x (θ 1 = j, θ 2 = k) Q j−1

s=1 f _x ¹

s−1 (x _s ) Q n t=j f _x ²

t−1 (x _t )

×(S _n (x ₀ , x ₁ , . . . , x _n )) ⁻¹ if j ≤ n < k, P _x (θ 1 = j, θ 2 = k) Q n

s=1 f _x ¹

s−1 (x _s ) Q k−1 t=j f _x ²

t−1 (x _t )

× Q n u=k f _x ³

u−1 (x _u )(S _n (x ₀ , x ₁ , . . . , x _n )) ⁻¹ if k ≤ n,

where

S n (x 0 , x 1 , . . . , x n )

=

n−1

X

j=1 n

X

k=j+1

n

p ^j−1 ₁ q 1 p ^k−j−1 ₂ q 2 j−1

Y

s=1

f _x ¹ _s−1 (x s )

k−1

Y

t=j

f _x ² _t−1 (x t )

n

Y

u=k

f _x ³ _u−1 (x u ) o

+

n

X

j =1

n p ^j−1 ₁ q ₁ p ^n−j ₂

j−1

Y

s=1

f _x ¹

s−1 (x _s )

n

Y

t=j

f _x ²

t−1 (x _t ) o + p ⁿ ₁

n

Y

s=1

f _x ¹

s−1 (x _s ).

We have

Π m,n+1 (x) = P _x (θ 1 = m, θ 2 > n + 1 | F n+1 )

= p 2 f _X ² _n (X n+1 )Π mn (x)S n (x 0 , x 1 , . . . , x n+1 )

× (S n+1 (x 0 , x 1 , . . . , x _n )) ⁻¹ and

S _n+1 (x ₀ , x ₁ , . . . , x _n+1 ) = H(X _n , X _n+1 , Π _n ¹ (x), Π _n ² (x))S _n (x ₀ , x ₁ , . . . , x _n ).

Hence

Π _m,n+1 (x) = p 2 f _X ² _n (X n+1 )Π mn (x) H(X n , X n+1 , Π _n ¹ (x), Π _n ² (x)) . By the above we have

ξ _mn (x) = P _x (θ 1 = j, θ 2 = k | F _mn )

=

p ^j−1 ₁ q 1 p ^k−j−1 ₂ q 2 j−1

Y

s=1

f _x ¹ _s−1 (x _s )

n−1

Y

t=j

f _x ² _t−1 (x _t )f _X ³ _n−1 (X _n ) S n (x 0 , x 1 , . . . , x n )

= q ₂ p 2

Π mn (x) f _X ³

n−1 (X n ) f _X ²

n−1 (X n ) .

We can observe that (X n , X n+1 , Π _n ¹ (x), Π _n ² (x), Π mn (x)) for n = m + 1, m + 2, . . . is a function of (X _n−1 , X _n , Π _n−1 ¹ (x), Π _n−1 ² (x), Π _m,n−1 (x)) and X n+1 . Moreover, the conditional distribution of X n+1 given F _n (cf. (10)) depends on X n , Π _n ¹ (x) and Π _n ² (x) only. These facts imply that {(X _n , X _n+1 , Π _n ¹ (x), Π _n ² (x), Π _mn (x))} ^∞ _n=m+1 form a homogeneous Markov process (see Chapter 2.15 of Shiryaev (1978)). This allows us to reduce the problem (4) for each m to the optimal stopping problem for the Markov pro- cess Z _m (x) = {(X n−1 , X _n , Π _n ¹ (x), Π _n ² (x), Π _mn (x)), m, n ∈ N, m < n, x ∈ E } with the reward function

h(t, u, α, β, γ) = q 2

p 2

γ f _t ³ (u)

f _t ² (u) .

Lemma 4. The solution of the optimal stopping problem (4) for m = 1, 2, . . . has the form

(11) σ ^∗ _m = inf



n > m : f _X ³ _n−1 (X n ) f _X ²

n−1 (X n ) ≥ R ^∗ (X _n )



where R ^∗ (t) = p 2

T

E r ^∗ (t, s)f _t ² (s) P _t (ds). Here r ^∗ = lim n→∞ r _n , where r 0 (t, u) = f _t ³ (u)/f _t ² (u) and

(12) r _n+1 (t, u) = max  f _t ³ (u) f _t ² (u) , p ₂

\

E

r _n (u, s)f _u ² (s) P _u (ds)

 . The function r ^∗ (t, u) satisfies the equation

(13) r ^∗ (t, u) = max  f _t ³ (u) f _t ² (u) , p 2

\

E

r ^∗ (u, s)f _u ² (s) P _u (ds)

 . The value of the problem is

(14) η m = E x (η m,m+1 | F m ) = q ₂ p 2

q ₁ p 1

Π _n ¹ (x)R ^∗ (X m ).

P r o o f. For any Borel function u : E × E × [0, 1] ³ → [0, 1] define two operators

T _x u(t, s, α, β, γ) = E _x (u(X _n , X n+1 , Π _n+1 ¹ (x), Π _n+1 ² (x), Π m,n+1 (x)) |

X _n−1 = t, X _n = s, Π _n ¹ (x) = α, Π _n ² (x) = β, Π _mn (x) = γ) and

Q x u(t, s, α, β, γ) = max{u(t, s, α, β, γ), T x u(t, s, α, β, γ)}.

By the well-known theorem from the theory of optimal stopping (see Shiryaev (1978), Ch. 2, and Nikolaev (1981)) we conclude that the solu- tion of (4) is the Markov time

σ _m ^∗ = inf{n > m : h(X n−1 , X _n , Π _n ¹ (x), Π _n ² (x), Π _mn (x))

= h ^∗ (X n−1 , X n , Π _n ¹ (x), Π _n ² (x), Π mn (x))}, where h ^∗ = lim k→∞ Q ^k _x h(t, u, α, β, γ). Then

T _x h(t, u, α, β, γ) = E _x  q 2

p ₂ Π m,n+1 (x) f _X ³ _n (X n+1 ) f _X ²

n (X _n+1 )    

X n−1 = t, X _n = u, Π _n ¹ (x) = α, Π _n ² (x) = β, Π _mn (x) = γ



= q 2

p 2

γp 2 E

 f _u ² (X n+1 ) H(u, X n+1 , α, β)

f _u ³ (X n+1 ) f _u ² (X n+1 )    

F n



(10) = q 2 γ

\

E

f _u ³ (s)

H(u, s, α, β) H(u, s, α, β) P _u (ds) = q 2 γ

and

(15) Q _x h(t, u, α, β, γ) = q 2

p ₂ γ max  f _t ³ (u) f _t ² (u) , p 2

 . Define r n (t, u) as in the statement of the lemma. We show that (16) Q ^l _x h(t, u, α, β, γ) = q ₂

p 2

γr l (t, u)

for l = 1, 2, . . . By (15) we have Q x h = (q 2 /p 2 )γr 1 . Assume (16) for l ≤ k.

By (10) we have

T _x Q ^k _x h(t, u, α, β, γ) = E _x  q 2

p 2

Π m,k+1 (x)r _k (X _n , X n+1 )    

X n−1 = t, X _n = u, Π _n ¹ (x) = α, Π _n ² (x) = β, Π _mn (x) = γ



= q 2

p 2

γp 2

\

E

r _k (u, s)f _u ² (s) P _u (ds).

It is easy to show (see Shiryaev (1978)) that

Q ^k+1 _x h = max{h, T _x Q ^k _x h} for k = 1, 2, . . .

Hence we get Q ^k+1 _x h = (q 2 /p 2 )γr k+1 and (16) is proved for l = 1, 2, . . . This gives

h ^∗ (t, u, α, β, γ) = q 2

p 2

γ lim

k→∞ r _k (t, u) = q 2

p 2

γr ^∗ (t, u) and

η _mn = ess sup

(τ,σ)∈T mn

E _x (ξ _τ,σ | F _mn )

= h ^∗ (X n−1 , X _n , Π _n ¹ (x), Π _n ² (x), Π _mn (x)).

We have

T _x h ^∗ (t, u, α, β, γ) = q 2

p 2

γp ₂

\

E

r ^∗ (u, s)f _u ² (s) P _u (ds) = q 2

p 2

γR ^∗ (u) and σ _m ^∗ has the form (11). By (5) and (10) we obtain

η _m (x) = f (X _n−1 , X _n , Π _n ¹ (x), Π _n ² (x)) = E(η _m,m+1 | F _m ) (17)

= E  q 2

p 2

Π _m,m+1 r ^∗ (X _m , X _m+1 )    

F _m

 (18)

= q ₂ p 2

Π mm

\

E

r ^∗ (X m , s)f _X ² _m (s) P X m (ds).

(19)

By Lemmas 4 and 3 the optimal stopping problem (6) has been trans- formed to the optimal stopping problem for the homogeneous Markov pro- cess

W = {(X m−1 , X _m , Π _m ¹ (x), Π _m ² (x)), m ∈ N, x ∈ E}

with the reward function

f (t, u, α, β) = q 1 q 2

p 1 p 2

· f _t ² (u)

f _t ¹ (u) αR ^∗ (u).

Lemma 5. The solution of the optimal stopping problem (6) for n = 1, 2, . . . has the form

(20) τ _n ^∗ = inf{k ≥ n : (X k−1 , X k , Π _k ¹ (x), Π _k ² (x)) ∈ B ^∗ } where B ^∗ = {(t, u, α, β) : f _t ² (u)/f _t ¹ (u) ≥ p 1

T

E v ^∗ (u, s) P _u (ds)}. Here v ^∗ (t, u)

= lim n→∞ v n (t, u), where v 0 (t, u) = R ^∗ (u) and (21) v n+1 (t, u) = max  f _t ² (u)

f _t ¹ (u) , p 1

\

E

v _n (u, s)f _u ¹ (s) P _u (ds)

 .

The function v ^∗ (t, u) satisfies the equation (22) v ^∗ (t, u) = max  f _t ² (u)

f _t ¹ (u) , p 1

\

E

v ^∗ (u, s)f _u ¹ (s) P u (ds)

 . The value of the problem is V n = v ^∗ (X n−1 , X n ).

P r o o f. For any Borel function u : E × E × [0, 1] ² → [0, 1] define two operators

T _x u(t, s, α, β) = E _x (u(X _n , X n+1 , Π _n+1 ¹ (x), Π _n+1 ² (x)) |

X n−1 = t, X n = s, Π _n ¹ (x) = α, Π _n ² (x) = β) and

Q _x u(t, s, α, β) = max{u(t, s, α, β), T _x u(t, s, α, β)}.

As in the proof of Lemma 4 we conclude that the solution of (6) is the Markov time

τ _m ^∗ = inf{n > m : f (X n−1 , X n , Π _n ¹ (x), Π _n ² (x))

= f ^∗ (X n−1 , X _n , Π _n ¹ (x), Π _n ² (x))}, where f ^∗ = lim _k→∞ Q ^k _x f (t, u, α, β). We have

T _x h(t, u, α, β) = E _x  q 1 q ₂ p 1 p 2

Π _n+1 ¹ (x) f _X ² _n (X n+1 )

f _X ¹ _n (X n+1 ) R ^∗ (X _n+1 )    

X _n−1 = t, X _n = u, Π _n ¹ (x) = α, Π _n ² (x) = β



= q 1 q 2

p 1 p 2

γp 1 E

 f _u ¹ (X n+1 )

H(u, X n+1 , α, β) · f _u ² (X n+1 )

f _u ¹ (X n+1 ) R ^∗ (X n+1 )    

F _n



(10) = q 1 q 2

p ₁ p ₂ αp 1

\

E

f _u ² (s)

H(u, s, α, β) H(u, s, α, β)R ^∗ (s) P _u (ds)

= q 1 q 2

p ₁ p ₂ αp 1

\

E

R ^∗ (s)f _X ² _n (s) P _{X n} (ds) and

Q x f (t, u, α, β) = q ₁ q ₂ p 1 p 2

α max  f _t ² (u) f _t ¹ (u) , p 1

\

E

R ^∗ (s)f _u ² P u (ds)

 (23)

= q ₁ q ₂ p 1 p 2

αv ₁ (t, u).

We show that

(24) Q ^l _x f (t, u, α, β) = q ₁ q ₂ p 1 p 2

αv _l (t, u)

for l = 1, 2, . . . , where v _l (t, u) is defined as in the statement of the lemma.

By (23) we have Q x f = ^q _p ^{1 q 2}

1 p 2 αv 1 . Assume (24) for l ≤ k. By (10) we have T _x Q ^k _x f (t, u, α, β) = E _x  q 1 q 2

p 1 p 2

Π _k+1 ¹ (x)v _k (X _n , X _n+1 )    

X n−1 = t, X _n = u, Π _n ¹ (x) = α, Π _n ² (x) = β



= q ₁ q ₂ p 1 p 2

αp ₁

\

E

v _k (u, s)f _u ¹ (s) P _u (ds).

Hence Q ^k+1 _x f = ^q _p ^{1 q 2}

1 p 2 αv k+1 and (24) is proved for l = 1, 2, . . . This gives f ^∗ (t, u, α, β) = q 1 q 2

p 1 p 2

α lim

k→∞ v k (t, u) = q 1 q 2

p 1 p 2

αv ^∗ (t, u) and

V _m = q ₁ q ₂ p 1 p 2

Π _m ¹ v ^∗ (X _m−1 , X _m ).

We have

T x f ^∗ (t, u, α, β) = q 1 q 2

p 1 p 2

αp 1

\

E

v ^∗ (u, s)f _u ¹ (s) P u (ds).

It follows that τ _n ^∗ has the form (20). The value of the problem (6) and (3) is

v 0 (x) = E x (V 1 | F 0 ) = q 1 q 2

p 1 p 2

E x Π ₁ ¹ (x)v ^∗ (x, X 1 )

= q 1 q 2

p 2

\

E

v ^∗ (x, s)f _x ¹ (s) P x (ds).

By Lemmas 4 and 5 the solution of the problem D 00 can be formulated

as follows.

Theorem 4.1. The compound stopping time (τ ^∗ , σ ^∗ _τ ∗ ), where σ ^∗ _m is given by (11) and τ ^∗ = τ ₀ ^∗ is given by (20) is a solution of the problem D 00 . The value of the problem is

P _x (τ ^∗ < σ ^∗ < ∞, θ 1 = τ ^∗ , θ 2 = σ _τ ^∗ ∗ ) = q 1 q 2

p 2

\

E

v ^∗ (x, s)f _x ¹ (s) P _x (ds).

R e m a r k 1. The problem can be extended to optimal detection of more than two successive disorders. The distribution of θ 1 , θ 2 may be more general. The general a priori distributions of disorder moments lead to more complicated formulae, since the corresponding Markov chains are not homogeneous.

References

T. B o j d e c k i (1979), Probability maximizing approach to optimal stopping and its appli- cation to a disorder problem, Stochastics 3, 61–71.

T. B o j d e c k i (1982), Probability maximizing method in problems of sequential analysis, Mat. Stos. 21, 5–37 (in Polish).

Y. C h o w, H. R o b b i n s and D. S i e g m u n d (1971), Great Expectations: The Theory of Optimal Stopping , Houghton Mifflin, Boston.

G. H a g g s t r o m (1967), Optimal sequential procedures when more than one stop is re- quired , Ann. Math. Statist. 38, 1618–1626.

M. N i k o l a e v (1979), Generalized sequential procedures, Lit. Mat. Sb. 19, 35–44 (in Russian).

M. N i k o l a e v (1981), On an optimality criterion for a generalized sequential procedure, ibid. 21, 75–82 (in Russian).

A. S h i r y a e v (1978), Optimal Stopping Rules, Springer, New York.

K. S z a j o w s k i (1992), Optimal on-line detection of outside observations, J. Statist. Plann.

Inference 30, 413–422.

M. Y o s h i d a (1983), Probability maximizing approach for a quickest detection problem with complicated Markov chain , J. Inform. Optim. Sci. 4, 127–145.

Krzysztof Szajowski Institute of Mathematics Technical University of Wroc law Wybrze˙ze Wyspia´ nskiego 27 50-370 Wroc law, Poland E-mail: szajow@im.pwr.wroc.pl

Received on 15.3.1996