Switching regression models with non-normal errors

A C T A U N I V E R S I T A T I S L O D Z I E N S I S FO LIA O EC O N O M IC A 141, 1997

Krystyna Pruska*

SW ITCH IN G REG RESSIO N M OD ELS W ITH N O N -N O R M A L ER R O R S

Abstract. In this paper two form s o f switching regression m odels with non-norm al

errors are considered. The pseudo maximum likelihood method is proposed for the estim ation o f their param eters.

M onte C arlo experiments results are presented for a special switching regression model, too. In this research there are compared distributions o f param eters estim ators for different distributions o f errors. The error distributions are as follows: norm al, Student’s or Laplace’s. T he maximum likelihood m ethod (for the norm al errors) is applied to the estim ation. In most of the cases the estim ators distributions d o not differ significantly.

Key words: switching regression models, maxim um likelihood m ethod, pseudo m axim um likelihood m ethod.

1. IN T R O D U C T IO N

The switching regression m odels are special cases o f m odels with random coefficients. In this paper two forms of these m odels are considered. The pseudo m aximum likelihood m ethod is proposed for the estim ation. This m ethod is an alternative to the m aximum likelihood m ethod.

M onte Carlo experiments results are presented for a special switching regression m odel too. In this research there are com pared the distributions o f the param eters estim ators for different distributions of errors. The error distributions are the following: normal, Student’s or Laplace’s. The m aximum likelihood m ethod and the pseudo m aximum likelihood m ethod are applied to the estimation.

2. T H E SW IT C H IN G R EG R ESSIO N M O D EL

The switching regression is a m ethod o f describing the dependence of a certain variable on two or m ore sets of variables, when the probability o f determ ining the value of a variable explained by a defined group of explanatory variables is either know n or unknown.

In this paper we shall deal with a particular case o f the form of a switching regression m odel (see, e.g. Q u a n d t 1972, K i e f e r 1978, C h a r e m z a 1981, p. 94-97, T o m a s z e w i c z 1985, p. 442-446, P r u s k a 1987):

_ j x 'ua i + eit with probability Я ^ {x'2la 2 + e2t with probability 1 —Я or

dt = x i ,ctl + Elt,

s, = x'2ta 2 + e2t, (2.2)

y t = m in {dt, s,},

where ( T - the sample size), 0 < Я < 1 and Я can be either

know n or unknown. O ther symbols are as follows:

yt, dt, st - variables explained by the model (dt, s, can be observable

or not, y t is observable);

x u> x2t - column vectors o f the explanatory variables;

a 1; a 2 - colum n vectors o f the m odel’s structural param eters;

Elt, e2t - errors; random com ponents of the m odel; random variables

with null expectation and variances and a \ and a\ , respectively, such that cov(elt) E Z t ) = 0, cov(elt, elt) = 0, cov(£lt, e2t) = 0, cov(elt, e2t) = 0, for t Фх

and t, x — { 1 , T}.

The param eters of the m odels (2.1) and (2.2), are usually estim ated by the m axim um likelihood m ethod.

3. PSEU D O M A X IM U M L IK EL IH O O D M E T H O D

The pseudo m axim um likelihood (PM L) m ethod is an alternative to the m axim um likelihood (M L) m ethod. The idea of the P M L m ethod is the following (see G o u r i e r o u x et al. 1984).

Let us consider a m odel of the form:

where:

y t - ^-dimensional random vector of the variables explained by the model;

x, - /7-dimensional vector of the explanatory variables;

0o - ^-dim ensional param eters vector;

e, - ^-dim ensional random vector o f errors with zero expectation and k, p are positive integers, T is the sample size, g is

a non-linear function.

For the

estimation of the parameters

such values of

are

taken

is a /7-dimensional vector) for which function:

L(yt, x„ 0) = £ Inf[(y„ g(x„ 0)] (3.2)

»=1

called the pseudo m aximum likelihood function reaches its m axim um , where / is the density function (for continous distributions) or the probability function (for discrete distributions) of distribution which belongs to the linear exponential family.

A family o f /с-dimensional probability distributions, indexed by param eter

m ( m e R k), is called linear exponential if every element o f the family has

the density function (or the probability function), which can be written as

f ( u , m) = exp {A(m) + B(u) + C(m)u} (3.3)

where u e R k, A(m), B(u) are scalars, C(m) is a row ^-dim ensional vector, and m is expectation of the distribution determined by f ( u , m) (see G o u - r i e r o u x at al. 1984).

D istributions such as binomial, negative binomial, Poisson’s, gam m a, norm al and m ultidim ensional norm al belong to this family.

The PM L m ethod leads to consistent and asymptotically normal estimators o f param eters o f investigated random variable (see G o u r i e r o u x et al. 1984).

4. A PPLIC A T IO N O F T H E PM L M E T H O D T O T H E ESTIM A TIO N O F SW IT C H IN G R EG R ESSIO N M O D EL

We can apply the PM L m ethod to the estim ation o f m odel (2.1), when we do not m ake any assum ptions about the class o f error distributions. We do n ot assume th at errors are norm al.

If /(и , m) in the form ula (3.2) is the density function o f the norm al distribution with expectation m and variance 1, then the P M L function for m odel (2.1.) has its m axim um , when the followig function has its m inimum:

T

Цу ,\ a 1; ot2, a u a z, X) = £ [j>( - E(yt)]2 (4.1)

(4.2) where is the density of у, with probability X and f 2 is the density o f y t with probability 1 — X. Next we can write:

In this case the PM L m ethod is reduced to the least squares non-linear m ethod.

I f we want to apply the PM L m ethod to the estim ation o f m odel (2.2), we have to write this m odel in another form (see P r u s k a 1992). Let us consider the model:

d, = X ' u t t ! + ( T 1 U l t ,

st = x'2ta2 + a zu 2t, (4.5)

yt = E(min{dt, st}) + Et

where uu and u2t are independent random variables with normal distributions, the expectation of which is equal to zero and the variance is equal to one, e, has unknow n distribution with the expectation equal to zero. Now, we can minimize the function

In this minimizing we can apply numerical approxim ation o f expectation

E(yt) using M onte C arlo m ethods (see L a r o q u e , S a l a n i e 1989).

E(yt) = Xx'u u l + (1 — X)x'Ztaz _(4.3)

And so we have:

T

E(yt> 0^2, f f j , o 2, Я) — \yt (Ях^{й| + (1 X)x2tcc2)y]2 _(4.4) t=i

T

U y t; a u a 2, а и a 2, X) = ^ [yt - E(yt)]2 _(4.6)

where

E(yt) = -

Eiminfxi,«! +

+ <r2w2t})

5. M O N T E C A R LO EX PE R IM E N TS

M onte Carlo experiments have, as their purpose, com parisons o f es-tim ators’ distributions for different distributions o f errors, when we apply

the m axim um likelihood m ethod (for the norm al errors) to the estim ation o f m odel (2.2). We consider three distributions of errors: norm al, Student’s and Laplace’s.

F o r all the cases the model has the form:

dt = x'lta 1 + ß l + Elt,

s, - x'2ta2 + ß2 + e 2„ (5.1)

y, = m in {d„ s,}

with u l — 1, ß i = —5,

a2

x 2t ~ iV(20; 5); symbol x ~ N(n, er) m eans that jc has the norm al distribution

with expectation ц and variance a 2.

W e generate samples for elt and e 2i with different distributions. The variances o f these distributions are equal: a2 for e u and a \ for e 2v We com pute the values o f d„ st, y t, and we determ ine M L-estim ates o f param eters а х, a 2, ß u ß 2, crl , a 2 for the norm al errors. Next we apply the same algorithm to estimating the param eters o f m odel (5.1) with Student’s errors and Laplace’s errors. We generate samples with size n = 20, 30, 50, 100. We repeat it ten times.

We set three hypotheses:

- param eter estim ator distributions do not differ if errors have the norm al distribution and the Student’s distribution;

- param eter estim ator distributions do not differ if errors have the norm al distribution and the Laplace’s distribution;

- param eter estim ator distributions do not differ if errors have the S tudent’s distribution and the Laplace’s distribution.

We apply the run test. In this test the critical value is к = 6 for significance level

a

Values к of the run test statistics are in the Tab. 1. In five cases we reject some hypotheses (к ^ fcj. In other cases we cannot reject hypotheses

(k > k j . These results do not allow to notice significant differences between

the param eter estim ators’ distributions for different errors distributions.

6. F IN A L R EM A R K S

In this paper the pseudo m aximum likelihood m ethod for the estim ation o f two forms of switching regression m odels with non-norm al errors is presented. This m ethod leads to consistent and asym ptotically norm al estim ators of m odel param eters. These properties are like properties of the m axim um likelihood estim ators but we do not have to assume th at m odel errors are norm al. It seems th at the PM L m ethod can find wide application

T a b l e 1 N um ber к o f runs in the ru n test sequences o f estimates

o f switching regression models param etres

Size of sample Com pared distribution o f errors Param eter “ i ßi «2 P2 a2 20 norm al d . and Student’s d . 15 11 14 13 9 12 norm al d . and Laplace’s d . 7 5 9 11 13 11 Student’s d. and Laplace’s d . 7 7 8 7 13 13 30 norm al d . and Student’s d. 9 11 12 14 7 10 norm al d . and Laplace’s d. 12 13 14 15 9 10 S tudent’s d . and Laplace’s d. 12 12 13 10 8 4 50 norm al d. and S tudent’s d . 11 13 13 13 7 12 norm al d . and Laplace’s d. 10 13 8 4 5 8 S tudent’s d. and Laplace’s d . 10 14 10 10 13 12 100 norm al d . and Student’s d. 11 11 8 10 7 9 norm al d . and Laplace’s d. 13 7 7 12 7 6 S tudent’s d . and Laplace’s d . 10 8 7 8 14 8

in switching regression analysis. M onte Carlo experiments do not allow to notice significant differences between the M L estim ator distributions for different errors distributions. These experiments were perform ed for one m odel only. We cannot generalize their results. In the future it is necessary to perform m ore experiments for different m odels and for different errors distributions.

R EFE R E N C E S

C h a r e m z a W. (1981): Ekonometryczne modele nierównowagi. Problemy specyfikacji i estymacji, „Zeszyty N aukow e Uniwersytetu G dańskiego” , G dańsk.

G o l d f e l d S., Q u a n d t R. (1981): Econometric Modelling with Non-normal Disturbances, „Journal o f Econom etrics” , 17, p. 141-155.

G o u r i e r o u x C., M o n f o r t A., T r o g n o n A . (1984): Pseudo M axim um Likelihood

Methods: Theory, „E conom etrica” , 52, p . 681-700.

K i e f e r N. (1978): Discrete Parameter Variation: Efficient Estimation o f a Switching Regression

Model, „Econom etrica” , 46, p. 427-434.

b a r o q u e G. , S a l a n i e B . (1989): Estimation o f M ulti-M arket Fix Price Models: An

Application o f Pseudo M axim um Likelihood Methods, „Econom etrica” , 57, p . 831-860.

P r u s k a К . (1987): Zastosowanie metody największej wiarygodności i regresji przełącznikowej

do estymacji ekonometrycznych modeli nierównowagi, D o cto r’s dissertation, Łódź.

P r u s k a K. (1992): On possibilities o f Applying Pseudo M axim um Likelihood M ethod to

Disequilibrium Models Estimation, Prace Instytutu Ekonom etrii i Statystyki Uniwersytetu

Łódzkiego, Seria D , 96.

Q u a n d t R. (1972): A New Approach to Estimating Switching Regressions, „Journal o f the A merican Statistical A ssociation” , 67, p. 306-310.

T o m a s z e w i c z A. (1985): Jednorównaniowe modele ekonometryczne p rzy nieklasycznych

założeniach, W U Ł, Łódź.

Krystyna Pruska

M O D E L E R EG R ESJI PR ZE Ł Ą C Z N IK O W E J ZE S K ŁA D N IK A M I LOSOW YM I O R O Z K ŁA D A C H R Ó ŻN Y C H O D N O R M A L N E G O

W pracy tej rozważane są dwie postacie modeli regresji przełącznikowej ze składnikam i losowym i o rozkładach różnych od norm alnego. D o estymacji p aram etró w tych m odeli zaproponow ana jest m etoda największej pseudowiarygodności.

Przedstawione są tu także wyniki eksperymentów M onte C arlo dla szczególnego modelu regresji przełącznikowej, dotyczące porów nania rozkładów estym atorów param etrów przy różnych rozkładach składników losowych (rozkład norm alny, Studenta i Laplace’a). W większości przypadków nie zauważa się statystycznie istotnych różnic między rozkładam i estym atorów , gdy stosujem y d o estymacji ten sam algorytm estymacyjny, wynikający z m etody największej wiarygodności przy założeniu, że b ad an a zm ienna m a rozkład norm alny.