MINIMAX MUTUAL PREDICTION

S. T R Y B U L A (Wroc law)

MINIMAX MUTUAL PREDICTION

Abstract. The problems of minimax mutual prediction are considered for binomial and multinomial random variables and for sums of limited random variables with unknown distribution. For the loss function being a linear combination of quadratic losses minimax mutual predictors are determined where the parameters of predictors are obtained by numerical solution of some equations.

1. Introduction. Suppose that a number of statisticians are observ- ing some random variables. Assume that the ith statistician is observing a random variable X i . He wants to predict the random variables of his part- ners. Let d ij (X i ) be the predictor he applies to predict X j . Let the loss connected with this prediction be L ij (X j , d ij (X i )). Then the total loss of all statisticians is

(1) L(X, d) =

l

X

i,j=1 i6=j

L ij (X j , d ij (X i ))

where X = (X 1 , . . . , X l ),

d =







− d 12 . . . d 1l

d 21 − . . . d 2l

. . . . d l1 d l2 . . . −





 =: [d ij ] ^l ₁ .

Suppose that the random variable X has distribution depending on an un-

2000 Mathematics Subject Classification: Primary 62F15.

Key words and phrases: minimax mutual predictor, binomial, multinomial, Bayes.

[437]

known parameter µ. The risk function is defined as (2) R(µ, d) = E µ (L(X, d)) =

l

X

i,j=1 i6=j

E µ (L ij (X j , d ij (X i )))

where E µ (·) denotes the expected value.

A mutual predictor d 0 is minimax if sup

µ

R(µ, d 0 ) = inf

d

sup

µ

R(µ, d).

In this paper we shall find minimax mutual predictors in some situations.

2. Mutual predictors for binomial random variables. Let the variables X 1 , . . . , X l be independent and have binomial distributions

f i (x i | p) = n i

x i



p ^x

(1 − p) ⁿ

^−x

where the f i are densities with respect to the counting measure. Let the losses L ij be quadratic. Then

L(X, d) =

l

X

i,j=1 i6=j

k ij (d ij (X i ) − X j ) ²

where k ij ≥ 0, P

i6=j k ij > 0. Hence the risk function R(p, d) can be repre- sented in the form

R(p, d) =

l

X

i,j=1 i6=j

k ij E p (d ij (X i ) − X j ) ² (3)

=

l

X

i,j=1 i6=j

k ij [E p (d ij (X i ) − n j p) ² + n j p(1 − p)].

We shall look for minimax mutual predictors. Consider predictors of the form

(4) d ij (X i ) = n j

X i + α

n i + γ , α > 0, γ > 0.

In this case

(5) R(p, d) =

l

X

i,j=1 i6=j

k ij

 n ² _j E p

 X i + α n i + γ − p

 2

+ n j p(1 − p)



=

l

X

i,j=1 i6=j

k ij

 n ² _j

(n i + γ) ² (n i p(1 − p) + (α − γp) ² ) + n j p(1 − p)

 .

The risk R(p, d) will be constant if (6)

l

X

i,j=1 i6=j

k ij

 n ² _j

(n i + γ) ² (−n i + γ ² ) − n j



= 0

and (7)

l

X

i,j=1 i6=j

k ij

 n ² _j

(n i + γ) ² (n i − 2αγ) + n j



= 0.

For γ = 0 the left side of the first equation is negative, and as γ → ∞ it tends to

(8) A =

l

X

i,j=1 i6=j

k ij n j (n j − 1) ≥ 0.

Moreover it is an increasing function of the parameter γ. Therefore if A > 0 there always exists a unique solution of equation (6).

Suppose that there exists a solution γ of (6). In this case there exists a solution α of (7) and

(9) α = γ/2.

Equation (6) can be solved numerically.

When n 1 = . . . = n l =: n > 1, the solution of (6) is

(10) γ = n

n − 1 ( √

2n − 1 + 1) and it is independent of k ij .

When all k ij = 0 except, say k 12 , equation (6) has a solution

(11) γ = n 1

n 2 − 1

 n 2

r 1 n 1

+ 1 n 2

− 1

n 1 n 2

+ 1

 .

Suppose that there exists a solution γ > 0 of (6). It is easy to prove that the predictors given by (4) are Bayes with respect to the a priori distribution of the parameter p given by the density

(12) g(p) = 1

B(α, γ − α) p ^α−1 (1 − p) ^γ−α−1 for 0 < p < 1.

Thus the mutual predictor d = [d ij ] ^l ₁ defined by (4), being a constant risk

Bayes predictor, is minimax if α = γ/2, where γ is a solution of (6).

When A = 0 a minimax mutual predictor is given by

(13) d ij (X i ) = n j /2.

It is obtained by letting γ → ∞, α = γ/2 in (4).

The problem of minimax prediction when only one k ij > 0 was solved by Hodges and Lehmann in [1].

When A > 0 the minimax risk is

(14) R(p, d 0 ) = 1

4

l

X

i,j=1 i6=j

k ij n ² _j (n i + γ) ² γ ² where d 0 satisfies (4), (6) and (9).

When A = 0 the minimax risk can be obtained by letting γ → ∞ in formula (14):

(15) R(p, d 0 ) = 1

4

l

X

i,j=1 i6=j

k ij n ² _j

where d 0 is given by (13).

3. Minimax mutual predictors for multinomial random vari- ables. Let now X i = (X i1 , . . . , X ir ), i = 1, . . . , 1, be independent random variables distributed according to multinomial laws

f i (x i | p) = n i !

x i1 ! . . . x ir ! p ^x ₁

. . . p ^x _r

,

where x i = (x i1 , . . . , x ir ) is the value of X i . Let the loss function be of the form

(16) L(X, d) =

l

X

i,j=1 i6=j

r

X

k=1

k ij (d ^(k) _ij (X i ) − X jk ) ² .

Let us consider the predictors (17) d ^(k) _ij (X i ) = n j

X ik + α k

n i + γ , i, j = 1, . . . , l, i 6= j, k = 1, . . . , r.

It is easy to show that for the loss function given by (16) with some k ij > 0 these Bayes predictors satisfy the equations

(18)

r

X

k=1

d ^(k) _ij (X i ) = n j , j = 1, . . . , l.

All these equations will surely be satisfied by d ^(k) _ij given in (17) when

(19) α k = γ/r, k = 1, . . . , r.

For predictors satisfying (17) and (19) the risk function will take the form (20) R(p, d) =

l

X

i,j=1 i6=j

r

X

k=1

k ij

 n ² _j

(n i + γ) ² (n i p k (1 − p k ) + (α k − γp _k ) ² ) + n j p k (1 − p k )

 . Notice that in (20) the coefficients of p ² _k for k = 1, . . . , r are the same.

They are zero when γ satisfies (6).

Let α k satisfy (19). In this case for γ given by (6) the expresion (20) will take the form

R(p, d) =

l

X

i,j=1 i6=j

k ij

 n ² _j (n i + γ) ²



n i − γ ² r

 + n j

 (21)

(6) =

l

X

i,j=1 i6=j

k ij

 n ² _j (n i + γ) ²



n i − γ ² r



+ n j + n ² _j

(n i + γ) ² (−n i + γ ² ) − n j



= r − 1 r

l

X

i,j=1 i6=j

k ij

n ² _j (n i + γ) ² γ ² .

From the above it follows that d = [(d ⁽¹⁾ _ij , . . . , d ^(r) _ij )] ^l ₁ , where d ^(k) _ij are given by (17), (19) and (6), is a constant risk mutual predictor. Let p = (p 1 , . . . , p r ) be a random variable. The expression

(22) E(E p (d ^(k) _ij (X i ) − X jk ) ² ) = E(E p (d ^(k) _ij (X i ) − n j p k ) ² + n j p k (1 − p k )) attains its minimum when

(23) d ^(k) _ij (X i ) = n j E(p k | X _i ) = n j E(p k | (X _i1 , . . . , X ir )).

Here E(p k | X _i ) denotes the conditional expectation of the random variable p k under the condition that X i is given. For the a priori distribution given by the density

(24) g(p 1 , . . . , p k ) = Γ (γ)

[Γ (γ/r)] ^r (p 1 . . . p r ) ^γ/r−1 we find that

(25) d ^(k) _ij (X i ) = n j E(p k | X _i ) = n j

X ik + γ/r

n i + γ

is a Bayes predictor.

We have shown that the mutual predictor d given by (17) and (19) is a Bayes predictor. Thus for γ satisfying (6) it is a minimax predictor for the loss function (16).

Equation (6) has a solution γ when A > 0 (see (8)). When A = 0 it is easy to show that a minimax mutual predictor is independent of X = (X 1 , . . . , X l ) and is given by

(26) d ^(k) _ij (X i ) = n j /r, i, j = 1, . . . , l, i 6= j, k = 1, . . . , r, and

(27) R(p, d) = r − 1

r

l

X

i,j=1 i6=j

k ij n ² _j .

If only one k ij 6= 0, the results of this section follow from the paper of Wilczy´ nski [4].

4. Minimax predictors of limited random variables. Suppose that the ith statistician is observing n i random variables X i1 , . . . , X in

with values in the interval [0, 1] and let X i = P n

k=1 X ik , X = (X 1 , . . . , X l ), where the X ik are independent. The statistician wants to predict the random variables X j , j = 1, . . . , l, j 6= i. Let the total loss function of all statisticians be of the form

(28) L(X, d) =

l

X

i,j=1 i6=j

k ij (d ij (X i ) − X j ) ² .

Let the random variables X ik , i = 1, . . . , l, k = 1, . . . , n i , have the same distribution function F and let µ = E F (X ik ). The risk function is

R(µ, d) =

l

X

i,j=1 i6=j

k ij E F (d ij (X i ) − X j ) ² (29)

=

l

X

i,j=1 i6=j

k ij [E F (d ij (X i ) − n j µ) ² + E F (X j − n _j µ) ² ].

We look for minimax predictors.

Let us try predictors of the form

(30) d ij = n j

X i + α

n i + γ .

Then

R(µ, d) =

l

X

i,j=1 i6=j

k ij

n ² _j

(n i + γ) ² [E F (X i − n _i µ) ² (31)

+ (α − γµ) ² + E F (X j − n _j µ) ² ]

≤

l

X

i,j=1 i6=j

k ij

 n ² _j

(n i + γ) ² (n i µ(1 − µ)

+ (α − γµ) ² ) + n j µ(1 − µ)

 . Let equations (6) and (7) be satisfied. Then α = γ/2 and

(32) R(µ, d) = 1

4

l

X

i,j=1 i6=j

k ij

n ² _j

(n i + γ) ² γ ² =: c

and for any random variable X = (X 1 , . . . , X l ) satisfying the conditions given at the beginning of this section,

(33) R(µ, d) ≤ c.

Let X ik have two-point distribution

(34) P (X ik = 0) = 1 − p, P (X ik = 1) = p.

Then µ = p and equality holds in (31). From formulae (31)–(34) and the results of Section 2 it follows that for γ satisfying (6) and α = γ/2, the predictor d = [d ij ] ^l ₁ given by (30) is a minimax mutual predictor. This holds when A > 0, where A is defined in (8).

For A = 0 a minimax mutual predictor is given by the formula

(35) d ij (X i ) = n j /2.

When only one k ij 6= 0 the problem considered in this section was solved by Hodges and Lehmann in [1].

Analogous results can also be obtained for mutual prediction of sample cumulative distribution functions for a properly chosen loss function. For minimax estimation of cumulative distribution functions, see Phadia [2].

For minimax estimators and predictors of many parameters and random variables, see [3], [4].

References

[1] J. L. H o d g e s and E. L. L e h m a n n, Some problems in minimax point estimation,

Ann. Math. Statist. 21 (1950), 182–191.

[2] E. G . P h a d i a, Minimax estimation of cumulative distribution functions, Ann. Stat- ist. 1 (1973), 1149–1157.

[3] S. T r y b u l a, Some problems of simultaneous minimax estimation, Ann. Math. Stat- ist. 29 (1958), 245–253.

[4] M. W i l c z y ´ n s k i, Minimax estimation for multinomial and multivariate hypergeo- metric distribution, Sankhy¯ a, Ser. A 47 (1985), 128–132.

Stanis law Trybu la Institute of Mathematics Technical University of Wroc law 50-370 Wroc law, Poland

Received on 30.11.1999;

revised version on 17.3.2000