INFERENCE ON THE LOCATION PARAMETER OF EXPONENTIAL POPULATIONS

INFERENCE ON THE LOCATION PARAMETER OF EXPONENTIAL POPULATIONS

Maria de F´ atima Brilhante Universidade dos A¸cores, DM, and CEAUL

Sandra Mendonc ¸a

Universidade da Madeira, CCEE, and CEAUL Dinis Duarte Pestana

Universidade de Lisboa, DEIO and CEAUL and

Maria Lu´ısa Rocha

Universidade dos A¸cores, DEG, CEEAplA and CEAUL

A token of friendship to Professor J. T. Mexia on his 70th birthday

Abstract

Studentization and analysis of variance are simple in Gaussian fa- milies because X and S

are independent random variables. We exploit the independence of the spacings in exponential populations with lo- cation λ and scale δ to develop simple ways of dealing with inference on the location parameter, namely by developing an analysis of scale in the homocedastic independent k-sample problem.

Keywords: studentization, analysis of scale, characterizations, independence of exponential spacings, location-scale families, F -ratio.

2000 Mathematics Subject Classification: 60E05.

∗

Research partially supported by FCT/OET.

1. Introduction

In what follows, X = (X

, X

, . . . , X

) denotes a random sample of size n

= n from the population X;

X = 1 n

X

X

and S

= 1 n − 1

X

(X

− X)

.

X

, . . . , X

, X

, . . . , X

, S

, . . . , S

k

– which for simplicity we shall denote S

, . . . , S

– will have similar meaning. Also for simplicity, we shall write n

= n

, j = 1, . . . , k, and N = n

+· · ·+n

the dimension of the combined sample. A pooled variance estimator

S

= (n

− 1)S

+ · · · + (n

− 1)S

N − k

will also be used.

The exact sampling distribution of functions of low empirical moments of Gaussian populations X

_ Gaussian(µ

, σ

) is easily derived, and hence we have a wealth of results to estimate, or to test, the location parameter, or to compare location parameters of several Gaussian populations, even for small samples, at least assuming homocedasticity.

For the one-sample and the homocedastic independent two-sample cases, Student (1908) devised how to get rid of the nuisance scale parameter in σ in

X − µ σ q

1 n

and in X

− X

− (µ

− µ

) σ q

1 n₁

+

2

,

respectively, dividing those standard Gaussian random variables by appro- priate functions of estimators of σ.

For the heterocedastic case, Smith (1936), Welch (1938), Satterthwaite

(1946) and Aspin (1948) established that

X

− X

− (µ

− µ

) s

σ

n

+ σ

n

has an approximate t

distribution, where the fractional number of degrees of freedom ν can be estimated by

˜ ν =

 s

n

+ s

n



s

n

(n

− 1) + s

n

(n

− 1)

.

Aspin (1949) tabulated high quantiles of this ‘fractional’ t distribution. Al- ternatively, inference on µ

− µ

can use random pairing of X

from X

with Y

chosen using simple random sampling without replacement from X

, cf. Sheff´e (1943, 1944), so that using the one-sample t-test for the ‘ran- dom’ differences D

= X

− Y

, k = 1, . . . , min(n

, n

) is in order.

Following David and Nagaraja (2003) we shall speak of studentiza- tion, in general populations, whenever we divide a random variable T = T (Θ(X), θ, ξ), which is a function of an estimator Θ(X) of the parameter of interest θ, depending both on this parameter and on a nuisance parameter ξ, by some appropriate function h(Ξ(X)) of an estimator ˜ ξ = Ξ(X) of the nuisance parameter, so that the ratio no longer depends on the nuisance pa- rameter ξ. Therefore, this ratio can be used as a fulcral variable, useful to construct interval estimators of θ; on the other hand, the above mentioned ratio is an adequate test statistic, under the null hypothesis H

: θ = θ

. We shall consider it external studentization when T and h(Ξ) are independent, and internal studentization otherwise.

The homocedastic independent k-sample problem, in Gaussian popula- tions, has been solved by Fisher (1925). Instead of getting rid of the nuisance parameter, Fisher’s ANalysis Of VAriance – in the sense of decomposing an estimator of the overall dispersion into components – uses the influ- ence of the location parameter estimates on the scale parameter estimator.

More precisely, as the variance σ

is the smaller second order moment, i.e., σ

< E[(X −A)

] for all A 6= E[X] (and, similarly, s

=

P

k=1

(x

−x)

<

1 n−1

P

k=1

(x

− a)

for all a 6= x =

P

k=1

x

), splitting the Total Sum of

Squares T SS = P

P

j=1

(X

− X)

, where X stands for the overall mean X =

P

i=1

P

j=1

X

of the combined sample X

e

= (X

, . . . .X

), into the Between Sum of Squares BSS = P

i=1

n

(X

− X)

and the Within Sum of Squares W SS = P

i=1

P

j=1

(X

− X

)

, we obtain two independent estimators of the variance, namely the means squares

and S

=

. Moreover, while S

is an unbiased estimator of σ

,

is unbiased under the null hypothesis µ

= µ

= · · · = µ

= µ, but when the alternative is true

is biased, and hence it must overestimate σ

, and the F -ratio

BSS k−1 W SS N−k

|H

_

F

should detect gross departures from the null hypothesis.

Once again, Welch–Satterthwaite pathbreaking techniques are useful in constructing approximate solutions without assuming homocedasticity (Satterthwaite, 1946; Welch, 1951; Oehlert, 2000).

Inference on location and scale in Gaussian populations is thus simple for two main reasons:

1. If X is a random sample from X _ Gaussian(µ, σ), X and S

are independent random variables, and thus in the two-sample setting, X

− X

and S

are independent random variables. Thus deducing the exact distribution of the studentized variables

X − µ S

r 1 n

and X

− X

− (µ

− µ

) S

r 1 n

+ 1

n

(Student, 1908) has been a simple task.

Unfortunately, independence of X and S

is a characterization of the Gaussian populations, and studentization in other populations, e.g.

uniform populations, is a hard task, even for samples of size 3 (Perlo, 1933).

2. Assume that we want to estimate the location parameter θ of an abso-

lutely continuous population X using the maximum likelihood estima-

tor ˆ θ. Under what conditions is ˆ θ = X?

Given a sample x = (x

, . . . , x

), ˆ θ must satisfy P

k=1

g

(ˆ θ − x

) = 0, where g(θ − x

) = ln [f (x

| θ)].

The special case x

= · · · = x

= 0 and x

= nu implies that (n−1)g

(u)+g

((1−n)u) = 0, n = 2, 3, . . . ; the case n = 2 shows at once that g must be an odd function. Thus g

(nu) = ng

(u), u ∈ R, n = 2, 3, . . . , and henceforth g

is a linear function. From this, g

(u) = Cu =⇒ g(u) =

2

C u

+ b, satisfying the condition R

R

f (x|θ) du = 1.

Then

f (x | θ) = r

α 2 π e

2α(x−θ)²

I

(x), α > 0.

This observation (Gauss, 1809) shows that the Gaussian family is unique, in the realm of absolutely continuous distributions, in having

X as the maximum likelihood estimator of its location parameter θ = E[X].

In the sequel we shall investigate how a characterization of the exponen- tial populations – namely that S

:= X

and the spacings S

:= X

− X

, k = 2, . . . , n are independent – can be used to establish simple re- sults in what concerns inference the location parameter λ of Y = δX + λ _ Exponential(λ, δ), where X _ Exponential(1) is the standard exponential, with probability density function f

(x) = e

I

(x).

2. Inference on the location

of exponential random variables

2.1. The one-sample case

Let y be a random sample of size n from Y _ Exponential(λ, δ), λ ∈ R, δ > 0, i.e., with distribution function

F

(y) =



1 − exp



− y − λ δ



I

(y).

The maximum likelihood estimators of the parameters are b λ = Y

and b δ = Y − Y

. From those we can easily construct unbiased estimators of λ and of δ, e λ = Y

−

and e δ =

P

k=1

(Y

− Y

). Moreover,

bδ = Y − Y

= 1 n

X

(Y

− Y

) = 1 n

X

(n + 1 − k)S

,

where the spacings S

= Y

− Y

_ Exponential 

0,



, k = 2 , . . . , n are independent and independent of S

= Y

− λ _ Exponential

0,

, cf. David and Nagaraja (2003).

Thus the studentized random variable

Y

− λ b δ =

Y

− λ δ δ b

δ

is the quotient of the independent random variables Y

− λ

δ _ Exponential (0,

) and b δ

δ _ Gamma n − 1, 0,

n

.

Therefore, the probability density function of Y

− λ δ b is

f

b δ

(x) = n − 1

(1 + x)

I

(x), i.e.,

Y

− λ b

δ _ P areto(0, n − 1)

can be used either to construct confidence intervals for the location parame- ter λ, or to test some specific hypothesis about where its value lies. Observe that the quantiles of this distribution are x

= α

− 1.

2.2. The ‘homocedastic’ independent two samples case

Although Johnson et al. (1995, p. 193) and Brilhante and Kotz (2008)

consider more general asymmetric Laplace random variables, herein we shall

consider only the simpler situation of equal sample sizes, leading to the usual

Laplace, obtained as X

− X

, where X

and X

are iid exponential random

variables.

Let Y

be a random sample of size

(n > 2 even) from Y

_ Exponential (λ

, δ), Y

a random sample of size

from Y

_ Exponential(λ

, δ) – hence we are assuming a ‘homocedastic’ situation δ

= δ

= δ –, and Y

and Y

independent. We shall denote Y

the k-th order statistics of Y

, Y

the j-th order statistics of Y

.

Using arguments analogous to those stated in the one sample case, it is easy to establish that the studentized random variable

W = Y

2

− Y

2

− (λ

− λ

)

δ b =

Y

2

− λ

− (Y

− λ

) δ

b δ

δ

,

where the random variables

Y

2

− Y

2

− (λ

− λ

)

δ _ Laplace

 2 n



and

bδ

δ =



Y

− Y

2

 + 

Y

− Y

2



2 _ Gamma



n − 2, 1 n



are independent.

In the general case, with eventually unequal sample sizes n

and n

, the adequate pooled estimator of the dispersion parameter δ is

b δ = n



Y

− Y

 + n

 Y

− X



n

+ n

.

From this we readily obtain the probability density function of W ,

f

(x) =

n 2

− 1 2 

1 +



I

(x)

and the upper quantiles

w

= 2 

(2α)

− 1 

, α < 1

2

(where n is the combined sample size).

2.3. The ‘homocedastic’ independent k samples case Let

Y

= (Y

, . . . , Y

) .. .

Y

= (Y

, . . . , Y

)

be independent random samples from k populations from Y

_ Exponential (λ

, δ), i = 1, . . . , k; j = 1, . . . , n. Once again, herein we shall only deal with the simplified problem of a balanced design, with all sample sizes equal.

For simplicity, we denote

λ the average

P

i=1

λ

, and we assume that the Y

, i = 1, . . . k, have been ordered so that λ

≤ λ

≤ · · · ≤ λ

; henceforth Y

 Y

 · · ·  Y

, where W  Z means that W is stochastically not greater than Z.

We shall use the notations Y

= min

Y

, i = 1, . . . , k, j = 1, . . . , n, Y

= min

Y

, i = 1, . . . , k, and more generally Y

the k−th order statis- tic from the i−th random sample.

We now split the Total Sum of Spacings T SSp into a Between Sum of

Spacings BSSp and a Within Sum of Spacings W SSp:

T SSp = X

X

(Y

− Y

) = BSSp + W SSp

= X

i=1

n 

Y

− Y

 +

X

X

 Y

− Y

 . The simple observation that

W SSp = X

i=1

X



Y

− Y



= X

X

(n + 1 − j) h

Y

− Y

i

| {z }

Gamma(n−1, δ)

| {z }

Gamma(N −k, δ)

shows that W SSp

N − k is an unbiased estimator of δ, either under H

or under the alternative H

.

Under H

, BSSp _ Gamma(k − 1, δ), and therefore BSSp

k − 1 is an un- biased estimator of δ. However, under H

, BSSp

k − 1 is a biased estimator of δ.

As a consequence, an ANOSp (ANalysis Of Spacings) table similar to the one-way ANOVA table

Sum of Spacings d.f. δ estimator F − ratio BSSp =

X

n h

Y

− Y

i

k − 1 BSSp k − 1

BSSp k−1 W SSp

N−k

|H

_

F

W SSp = X

i=1

X

h

Y

− Y

i

N − k W SSp N − k

should be able to detect gross departures from the null hypothesis.

3. Alternative results on external and internal studentization in exponential populations

The broad concept of studentization as defined in David and Nagaraja (2003) inspires alternative approaches to inference on the location parameter of an exponential population Y _ Exponential(λ, δ). Brilhante et al. (2001) summarize previous work by the authors, namely, in the one-sample case, the study of the studentized minimum using the range as the estimator of a simple function of the nuisance scale parameter,

Y

− λ Y

− Y

=

X

X

− X

,

with X =

is the standard exponential.

As Y

and Y

−Y

are independent, it is once again external studen- tization. Due to the memoryless of the exponential, X

−X

= X

, with probability density function f

(x) = (n−1) e

(1 − e

)

I

(x).

Using standard methods, the probability density function of W =

n:n−Y1:n

is

f

(x) = −n

"

B(n, nt) + nt ∂B(n, nt)

∂(nt)

#

I

(x)

= n B(n, nt)

n−1

X

k=1

nt

(n − k) + nt I

(x),

where B(p, q) = R

0

x

(1 − x)

dx, p, q > 0 is Euler’s integral of the first

kind.

Explicit formulas for the probability density function for n ≤ 30, and a table of high quantiles, can be requested from the authors. For larger va- lues, observe that n ln(n)

1:n

converges in distribution to a standard exponential random variables.

Internal studentization may be an interesting alternative; for instance,

τ

= Y

− λ Y

− Y

(1 ≤ i < k ≤ n) for appropriate choices of i and k has a smaller breaking point than any of the studentized variables considered so far. At first sight the problem looks analytically intractable, since Y

− λ and Y

− Y

have a very intricate dependence structure. But rewriting the above expression as

Y

− Y

= Y n τ

,

in view of Basu’s theorem, and with the standard notations for the Laplace transform L and inverse Laplace transforms L

at given points, we get that

L  y

f

(n−1;i,k)

(y); nx 

= Γ(n) Γ(n + 1 − i) n

Γ(k − i) Γ(n + 1 − k)

e

(1 − e

)

x

or

L  y

f

(n−1;i,k)

(y); x 

= Γ(n) Γ(n + 1 − i) n Γ(k − i) Γ(n + 1 − k)

e



1 − e



x

and hence

f

(n−1;i,k)

(y)

= Γ(n) Γ(n + 1 − i) n Γ(k − i) Γ(n + 1 − k)

1 y

L



 

e



1 − e



x

; y



  .

Choosing i and k so that

n i k

3j − 1 j − 1 2j

3j j 2j + 1

3j + 1 j + 1 2j + 2

we have the specially simple expression

f

(n−1)

(y)

= Γ(n)Γ(2j + 1) n Γ

(j)

1 y

L



  1 x



 e



1 − e

 x





j

; y



  .

Since

L



 e



1 − e



x ; y



 = 1 I(

)(y)

and

L

1 s

; y

!

= y

Γ(n) ,

remembering how convolution of densities and products of Laplace trans- forms are related we get explicit solutions such as, for n = 3, i=1, k = 3,

f

(x) =

 

 

 

 

 

 

 

 

 



0 x < 1

3 4(x −

3

)

3 x

1

3 ≤ x < 2 3

4

9 x

x ≥ 2

3 or, for n = 4, i = 2, k = 4,

f

(3)

(x) =

 

 

 

 

 

 

 

 

 



0 x < 1 4 3

32x

(4x − 1)

1

4 ≤ x < 1 2

3 (8x − 3)

32 x

x ≥ 1

2

.

References

[1] A.A. Aspin, An examination and further developments of a formula arising in the problem of comparing two mean values, Biometrika 35 (1948), 88–97.

[2] A.A. Aspin, Tables for use in comparisons whose accuracy involves two vari- ances, separately estimated, Biometrika 36 (1949), 290–293.

[3] M.F. Brilhante and S. Kotz, Infinite divisibility of the spacings of a Kotz-

Kozubowski–Podg´ orski generalized Laplace model, Statistics & Probability

Letters 78 (2008), 2433–2436.

[4] M.F. Brilhante, D. Pestana, J. Rocha and S. Velosa, Inferˆencia Estat´ıstica So- bre Localiza¸c˜ ao e Escala, Sociedade Portuguesa de Estat´ıstica, Ponta Delgada 2001.

[5] H.A. David and H.N. Nagaraja, Order Statistics, 3rd ed., Wiley, New York 2003.

[6] R.A. Fisher, Statistical Methods for Research Workers, Oliver and Boyd, Edin- burgh 1925.

[7] N.L. Johnson, S. Kotz and N. Balakrishnan, Continuous Univariate Distribu- tions, vol. 2, 2nd ed., Wiley, New York 1995.

[8] G.W. Oehlert, A First Course in Design and Analysis of Experiments, Free- man, New York 2000.

[9] V. Perlo, On the distribution of ‘Student’s’ ratio for samples of three drawn from the rectangular distribution, Biometrika 25 (1933), 203–204.

[10] D. Pestana, F. Brilhante and J. Rocha, The analysis of variance revisited, in Extreme Values and Additive Laws, Lisboa (1999), 73–77.

[11] D. Pestana and J. Rocha, An´ alise de escala – modelo exponencial, in A Es- tat´ıstica e o Futuro e o Futuro da Estat´ıstica, Salamandra, Lisboa (1993), 295–303.

[12] J. Rocha, Localiza¸c˜ ao e Escala em Situa¸c˜ oes n˜ ao Cl´ assicas, Disserta¸c˜ ao de Doutoramento, Universidade dos A¸cores, Ponta Delgada 1995.

[13] J. Rocha, Inference on location parameters – internally studentized statistics, Rev. Estat. (2001), 355–356.

[14] F.E. Satterthwaite, An approximate distribution of estimates of variance com- ponents, Biometrics Bulletin 2 (1946), 110–114.

[15] H. Scheff´e, On solutions of the Behrens-Fisher problem based on the t−distribution, Ann. Math. Stat. 14 (1943), 35–44.

[16] H. Scheff´e, A note on the Behrens-Fisher problem, Ann. Math. Stat. 15 (1944), 430–432.

[17] H. Smith, The problem of comparing the results of two experiments with unequal means, J. Council Sci. Industr. Res. 9 (1936), 211–212.

[18] Student, The probable error of the mean, (Reprinted in E.S. Pearson and

J. Wishart, (1958) “Student’s” Collected Papers, Cambridge Univ. Press,

Cambridge), Biometrika 6 (1908), 1–25.

[19] B.L. Welch, The significance of the difference between two means when the population variances are unequal, Biometrika 29 (1938), 350–361.

[20] B.L. Welch, On the comparison of several mean values: an alternative approach, Biometrika 38 (1951), 330–336.

Received 8 September 2009