SUFFICIENT CONDITIONS FOR THE STRONG
CONSISTENCY OF LEAST SQUARES
ESTIMATOR WITH α-STABLE ERRORS
João Tiago Mexia
Mathemati sDepartment, Fa ultyof S ien eandTe hnology
New Universityof Lisbon
Monte da Capari a2829516 Capari a, Portugal
e-mail: par rf t.unl.pt
and
JoAo˜ Lita da Silva
NewUniversity ofLisbon,Mathemati s Department
Fa ultyofS ien eandTe hnology
Quinta da Torre, 2825114 Monteda Capari a, Portugal
e-mail: jsf t.unl.pt
Abstra t
Let Yi = xTi β+ ei, 1 6 i 6 n, n >1 bealinearregressionmodel
andsupposethattherandomerrorse1, e2, . . .areindependentandα-
stable. In this paper, we obtain su ient onditions for the strong
onsisten y of the least squares estimator βe of β under additional
assumptionsonthenon-randomsequen ex1, x2, . . .ofrealve tors.
Keywords: linearmodels,leastsquaresestimator,strong onsisten y,
stability.
2000Mathemati sSubje t Classi ation: 60F15.
1. Introdu tion
Thestrong onsisten yoftheLeastSquaresEstimator(LSE)hasbeenstud-
iedonthe last yearsbymanyauthors (seefor instante[6, 10,11,12,13,14℄
or [15 ℄). In the papers [10℄ and [11℄ are obtained ne essary and su ient
onditions for the LSE strong onsisten y. Nevertheless, the formulation
used by the authors of the papers quoted above assumes an i.i.d. random
errorsequen e withnullexpe tedvalueandnite absolutemoment oforder
1 6 r < 2. In a rst stage, our purpose will be to derive LSE strong on-
sisten y from a formulation that is, insome sense, similar to the des ribed
previously. As a matter of fa t, we will assume that the errors e1, e2, . . .
arei.i.d. α-stableex luding any hypothesis ontheir absolute moments. Let
us point out thatthe stability onditionfor theerrorsis introdu ed here to
makeusefulthesuitable'linearity' propertiesofthestabledistributions. On
a se ond stage, we generalized the above assumptions supposing that the
random variables e1, e2, . . .areonly independent andα-stable, obtainingan
extension of theresultspresentedin[10 ℄ and[11 ℄.
Consider the linearmodel,
(1.1) Yi = xTi β+ ei, 1 6 i 6 n, n >1,
where x1, x2, . . . areknown non-random real κ-ve tors,
β:=
β1
.
.
.
βκ
istheve tor of(unknown)parameters andei istherandomerror inthei-th
observation (i = 1, . . . , n).
Setting
Sn:= x1xT
1 + . . . + xnxT
n
wewillassumethatS−1
n existsfornlargeenough(forsmallnsu hthatS−1
n
doesnot exist,S−1
n anbedened arbitrarily). DenotingtheLSEofβ byβe
we an write
(1.2) βe = β + S−1n
Xn
i=1
xiei
sothat, βe isstrongly onsistent ifand only ifS−1n Xn
i=1
xiei −→ 0a.s. .
2. Auxiliary tools
Thefollowing notational onventions willbe adoptedthroughout thepaper.
Given a ve tor x∈Rκ,the eu lidean ve tor norm will bedenoted by
||x|| :=√ xTx
and the matrix normof anmatrix A∈ Mκ×κ(R) willbeindi ated by
|||A||| := sup
x6=0
||Ax||
||x|| .
The tra e of A ∈ Mκ×κ(R) will be denoted by tr(A). For any matrix A∈ Mκ×κ(R)withrealeigenvalueswewillindi atebyνmax(A)andνmin(A)
the maximum and minimum eigenvalue of A respe tively. Given two sym- metri matri esA, B∈ Mκ×κ(R) we writeA > B to denotethat A− Bis
nonnegative denite;wewillusethenotationA> Btoindi atethatA− B
ispositive denite.
Itiswell-knownthatarandomvariableZ (oritsdistribution) issaid to have a stable distribution if for any positive numbers c1 and c2, there is a
positive numbera anda realnumber bsu h that
(2.1) c1Z1+ c2Z2 = a Z + bd
where Z1 and Z2 are independent opies of Z.1 A random variable Z is
alledstri tly stable if (2.1) holdswithb= 0 (see [16℄).
Equivalently (see [16 ℄), a random variable Z is said to have a stable
distribution ifthere are parameters 0 < α 6 2, σ >0, −1 6 λ 6 1 and µ
real su hthat its hara teristi fun tion hasthefollowing form,
(2.2)
E eiZt
=
exp − σα |t|α
1 − iλ(signt) tanπα 2
+ iµt
!
if α6= 1
exp − σ |t|
1 + iλ2
π (sign t) log |t|
+ iµt
!
if α= 1 ,
where
signt:=
1 if t >0 0 if t= 0
−1 if t <0 .
Sin e(2.2) is hara terized byfour (unique) parameters
α∈]0, 2], σ >0, λ∈ [−1, 1], µ∈R
respe tively,theindexofstability,thes aleparameter,theskewnessparame-
ter andtheshift parameter wewilldenotestabledistributionsbySα(σ, λ, µ)
and write
Z ∼ Sα(σ, λ, µ)
to indi ate that Z has the stable distribution Sα(σ, λ, µ). A stable random
variableZ withindex ofstability α is alledα-stable.
Let us remark that a random variable Z on entrated at one point is always stable, i.e., a onstant µ has degenerate distribution Sα(0, 0, µ) for
1
ThenotationX = Yd meansthattherandomvariablesXandY agreeindistribution, thatis,FX(x) = FY(x),x ∈R.
any 0 < α 6 2. This degenerate ase is of no spe ial interest and, unless
stated expli itly, we always assume that Z is non-degenerate. In fa t, de- generate distributions have unusualproperties, for example, all moments of
a degenerate distribution arenite, whereas a non-degenerate stable distri-
bution with0 < α < 2 hasinnite se ondorder moments.
The distribution fun tions of α-stable random variables are absolutely
ontinuous and their densitieshave derivatives of anyorder in every point,
but ex luding some spe ial ases, their representations an be expressed
onlythrough ompli atedspe ialfun tions. However,thereexistasymptoti
expansionsofα-stabledensitiesinaneighborhoodoftheoriginor ofinnity (see [9,17 ℄or [18℄). Themain spe ial aseis given by
S2(σ, λ, µ) = N (µ, 2σ2) (σ > 0).
In this situation the skewness parameter λ is totally irrelevant. Hen e, a
random variable
Z ∼ S2(σ, λ, µ) (σ > 0)
hasprobabilitydensityfun tion
fZ(z) = 1
2σ√πe−(z−µ)24σ2 , z∈R.
The remaining spe ial ases of probability densities of α-stable random
variables in losed form are: the Cau hy distribution S1(σ, 0, µ) whose
densityis
f(x) = σ
π (x − µ)2− σ2 , x∈R
and the LévydistributionS1/2(σ, 1, µ) whose densityis
f(x) =
r σ
2π(x − µ)3e−2(x−µ)σ , x > µ.
Somebasi properties ofstabledistributions are listedbelow: (see [16℄)
(1) Let Z1 and Z2 be independent random variables with Zi ∼ Sα(σi, λi, µi),i= 1, 2. Then
Z1+ Z2∼ Sα
(σα1 + σα2)1/α,λ1σα1 + λ2σα2
σα1 + σα2 , µ1+ µ2
.
(2) Let Z ∼ Sα(σ, λ, µ) and let a be a real onstant. Then Z + a ∼ Sα(σ, λ, µ + a).
(3) LetZ ∼ Sα(σ, λ, µ)and letabea non-zero real onstant. Then aZ∼ Sα(|a| σ,sign (a)λ, aµ) if α6= 1 aZ∼ S1
|a| σ,sign(a)λ, aµ − 2
πσλalog |a|
if α= 1.
(4) LetZ ∼ Sα(σ, λ, µ) withα6= 1. ThenZ is stri tly stableifand only
ifµ= 0.
(5) Z ∼ S1(σ, λ, µ)isstri tly stableifand onlyifλ= 0.
(6) Z ∼ Sα(σ, λ, µ)issymmetri about µifand onlyifλ= 0.
(7) LetZ have distribution Sα(σ, λ, 0) withα <2. Thenthere exist two
i.i.d. randomvariablesA andB with ommon distributionSα(σ, 1, 0)
su hthat
Z =d
1 + λ 2
1/α
A−
1 − λ 2
1/α
B if α6= 1
Z =d
1 + λ 2
A−
1 − λ 2
B+
+σ
1 + λ π
log
1 + λ 2
−
1 − λ π
log
1 − λ 2
if α = 1.
The next statement is a powerful tool in matrix analysis and generalizes a
result whi h is a sine qua non for probability theory: the s alar version of
the Krone kerlemma.
Lemma 2.1 Krone ker's lemma (matrix ase). Let xi, i = 1, 2, . . . be a
sequen e ofrealκ-ve tors,Ai asequen e ofreal symmetri nonsingularκ×κ
matri es with Ai+1 > Ai > O and Pi a sequen e of nonsingular κ × κ
matri es. Suppose that ri=P∞ j=iA−1
j xj exists (and isnite). Then:
(i) If A∞:= lim
n→∞
An existsandis nite, then lim
n→∞
A−1
n
Xn
i=1
xi existsand
isnite.
(ii) If lim
n→∞
1
tr(An) = 0 then lim
n→∞
1
tr (An) Xn
i=1
xi= 0.
(iii) If lim
n→∞
A−1
n = O, lim
n→∞
Pnrn= 0and
lim sup
n→∞
Xn
i=1
A−1n Ai− Ai−1
P−1
i
< ∞ then lim
n→∞A−1
n
Xn
i=1
xi = 0.
P roof. Theproof an be founded in[1 ℄on page 2.7.
Remark 1. The ondition lim supn→∞Pn
i=1
A−1n Ai− Ai−1
P−1
i
< ∞
isimplied bythemu h stronger ondition
lim sup
n→∞
νmax(An) νmin(An) <∞.
For some remarkson amatrix versionof Krone ker's lemma, see[2℄.
To nish this se tion, we present an important result that gives us the
almost sure onvergen e for sums of independent random variables Xi ∼ Sα(σi, λi, µi).
Proposition 2.1. Let {Xi, i > 1} be independent random variables Xi ∼ Sα(σi, λi, µi),0 < α 6 2. Then,
X∞
i=1
Xi onverge a.s. ⇐⇒
X∞
i=1
σαi <∞ and X∞
i=1
µi onverge.
P roof. We establish the proof only for 0 < α < 2 sin e the ase α = 2
is obvious in view of the form of hara teristi fun tions of normal law.
We show rst that if
P∞
i=1σαi <∞ and P∞
i=1µi onverge then P∞ i=1Xi
onverges almostsurely. Setting,
an:= (σα1 + . . . + σαn)1/α, bn:= λ1σα1 + . . . + λnσαn σα1 + . . . + σαn
and cn:= µ1+ . . . + µn
we get, forall n∈N,
Xn
i=1
Xi ∼ Sα(an, bn, cn)
provided thatXi∼ Sα(σi, λi, µi).
Therefore,we an write
(2.3)
Xn
i=1
Xi= ad nYn+ cn if α6= 1
and
(2.4)
Xn
i=1
Xi= ad nYn+ 2
π bnan log an+ cn if α= 1
where {Yn, n > 1} is a sequen e of random variables satisfying Yn∼ Sα(1, bn,0). Hen e, there exist two i.i.d. random variables An and Bn with ommon distribution Sα(1, 1, 0) su h that,
(2.5) Yn=d
1 + bn
2
1/α
An−
1 − bn 2
1/α
Bn if α6= 1
and
Yn=d
1 + bn 2
An−
1 − bn
2
Bn+1 + bn π log
1 + bn 2
−
−1 − bn
π log
1 − bn
2
if α= 1.
(2.6)
Sin e
P∞
i=1σαi <∞ and P∞
i=1µi is onvergent we obtain the
onvergen e inlawof Yn sin e An and Bn haveboth distribution Sα(1, 1, 0)
for all n ∈ N (let us note that bn is a onvergent sequen e). A ording
to (2.3) and (2.4) , the series
Pn
i=1Xi onverges in law whi h implies
the existen e of a (proper) random variable S su h that Pn
i=1Xi −→ Sa.s.
(see [4℄).
Wenowshowthat,if
P∞
i=1Xi onvergesalmostsurelythenP∞
i=1σαi <∞
and
P∞
i=1µi onverge. Sin e
|bn| =
λ1σα1 + . . . + λnσαn σα1 + . . . + σαn
61, ∀n ∈N
thereexistasubsequen e bηn
onvergingtoapointof[−1, 1]whi himplies
the onvergein lawofthe following random sequen es:
1 + bηn 2
1/α
Aηn−
1 − bηn
2
1/α
Bηn if α6= 1
and
1 + bηn 2
Aηn−
1 − bηn
2
Bηn+1 + bηn π log
1 + bηn 2
−
−1 − bηn
π log
1 − bηn
2
if α= 1 .
Therefore,thealmost sure onvergen e of
P∞
i=1Xi leads to
n→∞lim
ηn
X
i=1
σαi <∞
(see Theorem14.2 of[3℄ inpage 193)sothat,
X∞
i=1
σαi <∞
sin e this isa seriesof non-negative terms. Thus,
λ1σα1 + . . . + λnσαn+ . . .
σα1 + . . . + σαn+ . . . onverge,
X∞
i=1
µi onverge.
3. The strong onsisten y of LSE
Fromthesixties,thestrong onsisten yofLSEhasattra tedmu hattention
frommanystatisti ians. Inthemiddleandlateofseventiestheproblemwas
satisfa torily solved for the ase where the random errors possesses a nite
varian e. In fa t, a paper by Lai, Robbins and Wei (1979) showed that if
the e1, e2, . . . arei.i.d. withE e1
= 0and 0 < E e21
<∞ thena su ient
onditionfor the strong onsisten y ofβe is,
S−1n = Xn
i=1
xixTi
!−1
−→ O
as n → ∞. Hen e, the parti ular ase of the linear regression model (1.1)
givenbyi.i.d. stri tlystableerrorswithindexofstabilityα= 2is ompletely
under overbytheLai,Robbins &Wei'sresult.
Theorem 3.1. Consider the linear regression model (1.1) . If
(a) the random variables e1, e2, . . . are i.i.d. S2(σ, λ, 0),
(b) lim
n→∞S−1n = O,
then βe is strongly onsistent.
Remark 2. Letusnotethatthene essityofthe onditionlimn→∞S−1n = O
on the LSE strong onsisten y it had been proved earlier by H. Drygas
(1976) onthesameassumptions fortheerror sequen e: e1, e2, . . . i.i.d. with E e1
= 0 and 0 < E e21
<∞.
Moregenerally,
Theorem 3.2. Consider the linear regression model (1.1) . If
(a) the random variables e1, e2, . . . are i.i.d. S2(σ, λ, µ) withµ6= 0,
(b) lim
n→∞S−1n = O, lim sup
n→∞
Xn
i=1
S−1n xixTi
< ∞and
X∞
i=1
S−1
i xi <∞,
then βe is strongly onsistent.
P roof. From (1.2)weget
βe = β + S−1n Xn
i=1
xi(ei− µ) + µS−1n
Xn
i=1
xi.
Sin e ei − µ ∼ S2(σ, λ, 0) the thesis is a onsequen e of Theorem 3.1 and Krone ker's lemma (matrix ase).
The next theorem an be interpreted as an alternative path to the results
presented in[10℄ and [11℄. Indeed, the authors of thepapers quoted above
supposean i.i.d. error random sequen ewhere ea h term hasnull expe ted
value and nite absolute moment of order 1 6 r < 2. Our results will
establishes thestrong onsisten y of the LSE assuming only that the error
random sequen eisi.i.d. α-stable(α < 2).
Theorem 3.3. Let 0 < α < 2, α 6= 1 and onsider the linear regression
(a) the random variables e1, e2, . . . are i.i.d. Sα(σ, λ, µ),
(b) lim
n→∞S−1n = O, lim sup
n→∞
Xn
i=1
S−1n xixTi
<∞ and X∞
i=1
S−1i xi
min (α,1)<∞,
then βe is strongly onsistent.
P roof. Theidentity (1.2) guarantees,
βe = β + S−1n Xn
i=1
xiei = β + S−1n Xn
i=1
xie∗i + µ S−1n Xn
i=1
xi
withe∗i ∼ Sα(σ, λ, 0). Ontheonehand,byKrone ker's lemma(matrix ase) we have µ S−1n Pn
i=1xi −→ 0 as n → ∞ provided that
P∞i=1S−1
i xi
< ∞
and lim supn→∞Pn i=1
S−1n xixT
i
< ∞. On the other hand, ea h ompo-
nent ofthe ve tor
Pn i=1S−1
i xie∗i hasdistribution
Sα
σ
Xn
i=1
|aij|α
!1/α
, λ
Xn
i=1
sign(aij) |aij|α Xn
i=1
|aij|α
,0
, j= 1, . . . , κ
where aij isthej-th omponent of theve tor ai := S−1i xi. Sin e
X∞
i=1
|aij|α 6 X∞
i=1
S−1i xi
α
we obtain the almost sure onvergen e of
P∞ i=1S−1
i xie∗i through the
Proposition2.1and Krone ker's lemma (matrix ase) permitus to on lude
S−1
n
Pn
i=1xie∗i −→ 0a.s. .
On the next result we will use the notation |x|min := min |x1| , . . . , |xκ|
where xis annon-random κ-ve tor.
Theorem 3.4. Consider the linear regression model (1.1) . If
(a) the random variables e1, e2, . . . are i.i.d. S1(σ, λ, µ),
(b) lim
n→∞S−1n = O, lim sup
n→∞
Xn
i=1
S−1n xixTi
< ∞and
X∞
i=1
S−1i xi
log
S−1i xi
min onverges,
then βe is strongly onsistent.
P roof. Therelation (1.2) givesus
βe = β + S−1n Xn
i=1
xiei = β + S−1n Xn
i=1
xie∗i + µ S−1n Xn
i=1
xi
with e∗i ∼ S1(σ, λ, 0). Ea h omponent of the ve tor Pn i=1S−1
i xie∗i has
distribution
S1
σ
Xn
i=1
|aij| , λ
Xn
i=1
sign (aij) |aij| Xn
i=1
|aij|
,−2σλ π
Xn
i=1
aijlog |aij|
, j= 1, . . . , κ
where aij isthej-th omponent of ai := S−1i xi. Therefore,we have
S−1
n
Xn
i=1
xie∗i −→ 0a.s.
whi h is a onsequen e of the Proposition 2.1 and the Krone ker's lemma
(matrix ase) provided that limi→∞S−1
i xi = 0 (note that our assumptions guarantees
S−1i xixT
i
< ∞ as i→ ∞).
Sin e
X∞
i=1
S−1
i xi
<∞ and lim sup
n→∞
Xn
i=1
S−1n xixTi
< ∞
we get µ S−1n Xn
i=1
xi −→ 0 as n→ ∞ and thethesisis established.
Remark 3. If the random errors e1, e2, . . . are i.i.d. stri tly stable with
index of stability 0 < α < 2 then the strong onsisten y of the LSE is still valid assuming limn→∞S−1n = O, lim supn→∞Pn
i=1
S−1n xixT
i
< ∞
and
P∞ i=1
S−1i xi
α<∞.
The previous results an be extended to the ase where therandom varia-
bles e1, e2, . . . are (only) independent and ei ∼ Sα(σi, λi, µi). In this sense,
we nish this work with a possible generalization where the assumption of
identi aldistributionforrandomerrorse1, e2, . . .is ompletelyex luded. For
abetterpresentationofourideasandtheirasso iatedresultsletusstartwith
the ase0 < α 6 2,α6= 1.
Theorem 3.5. Let 0 < α 6 2, α 6= 1 and onsider the linear regression
model (1.1) . If
(a) the random variables e1, e2, . . . are independent su h that ei ∼ Sα(σi, λi, µi),
(b) lim
n→∞S−1n = O,lim sup
n→∞
Xn
i=1
S−1n xixTi
< ∞,
X∞
i=1
σi
S−1i xi
α
<∞
and
X∞
i=1
µiS−1
i xi <∞,
then βe is strongly onsistent.
P roof. From(1.2)weonlyhavetoprovethealmostsure onvergen eofthe series
P∞ i=1S−1
i xiei sin e the Krone ker's lemma (matrix ase) establishes the thesis. Ea h omponent of theve tor
Pn i=1S−1
i xiei hasdistribution,
Sα
Xn
i=1
σi|aij|α
!1/α
, Xn
i=1
σi|aij|α
λisign(aij) Xn
i=1
σi|aij|α
, Xn
i=1
µiaij
, j= 1, . . . , κ
whereaij isthej-th omponentofai:= S−1i xisothat,fromProposition2.1 we obtain thealmostsure onvergen eof
P∞ i=1S−1
i xiei.
The ase α= 1 isdes ribed next.
Theorem 3.6. Consider the linear regression model (1.1) . If
(a) the random variables e1, e2, . . . are independent and su h that ei ∼ S1(σi, λi, µi),
(b) lim
n→∞S−1n =O,lim sup
n→∞
Xn
i=1
S−1n xixTi
<∞,
X∞
i=1
σi
S−1i xi
log S−1i xi
min
onverges and
X∞
i=1
µiS−1
i xi <∞,
then βe is strongly onsistent.