A C T A U N I V E R S I T A T I S L O D Z I E N S I S
F O LIA O EC O N O M IC A 141, 1997
Krystyna Pruska*
ON T H E C L A SSIFIC A TIO N O F OBSERVATIONS IN T H E SW ITC H IN G R E G R E SSIO N
Abstract. T he paper discusses the method of determ ining the sample division indicator for the switching regression model in case o f tw o states generating values of the explained variable, which ensures the least risk o f making a mistake, understood as the expected value o f relevant loss function. This paper is an attem pt to take advantage o f the discrim ination analysis elements in the switching regression analysis. Key words: switching regression model, discrim ination analysis, loss function.
1. IN T R O D U C T IO N
The switching regression is a m ethod of describing the dependence of a certain variable on two or m ore sets of variables, when the probability o f determ ining the value of a variable explained by a defined group of explanatory variables is either know n or unknown. The analyzed relations are presented by m eans o f specific statistical models called the switching regression models.
The param eters of these m odels can be estimated by different m ethods. The maximum likelihood m ethod is applied for this purpose m ost frequently. It gives consistent, asymptotically m ost efficient and asymptotically norm al estim ators o f the switching regression m odels’ param eters (see K i e f e r
1978). A n im portant fact here is having inform ation through which the state o f setting the value of the explained variable is generated, i.e. which set o f the explantory variables determines this value. Very often such d ata are not available and decision is taken under uncertainty, on the basis of the value o f some random variable, which is subjectively chosen as being adequate fo r perform ing such a role. This variable can be called on
indicator of sample division. The way of classifying observations in the switching regression models affects the shape o f the likelihood function being the basis for determining estim ators of the m odel’s param eters.
The present paper suggests the m ethod o f determ ining the sam ple division indicator for the switching regression m odel in case of two states generating values o f the explained variable, which ensures the least risk of m aking a m istake, understood as the expectation of relevant loss function. This paper is an attem pt to take advantage of the discrim inant analysis elements in the switching regression analysis.
2. T H E BASIC PROBLEM O F D IS C R IM IN A T IO N
One of the problem s the discrim inant analysis is concerned with, is decision, basing on a random sample, on which o f the two possible classes o f the probability distribution the distribution of the investigated variable can be included into (see Z u b r z y c k i 1970, p. 294-299), when the probability o f which population a given sample element comes from can be either known or unknown. In this paper we shall deal with the case when this probability is known.
Let us assume th at we have a random sample X v ..., X n, selected from a population being a com bination (set sum) o f two populations. Let / x(.) and / 2(.) denote know n densities o f these populations, i.e. distributions of the examined feature o f given population. Let p L be probability th at a given sample element comes from the first population, and p 2 = 1 — p y p ro b a -bility th at it comes from the second population. Let L x denote a loss resulting from classifying the element o f random sample X l t X n into the second population, when in fact it comes from the first population, and L 2 - loss due to classifying a sample element into the first population, when in fact it comes from the second population ( L t and L 2 are know n values). F u rth er let A l and A 2 be such sets o f real num bers which are disconnected; in total they give a set of all real num bers and A i is the set o f these values for which we conclude that a given sample element comes from the first population, and A 2 is the set of these values, for which we decide th a t a given sample element comes from the second population. Sets A t and A 2 can be defined in different ways, depending on the criterion determ ining the principles o f decision m aking o f classifying an observation into a specific observation. In the classical discrim ination analysis sets A t and A 2 are determ ined in such a way so as to minimize the risk, i.e. the loss expected value (loss function) resulting from the way o f m aking decision o f observation classifying would be m inimum .
To facilitate further considerations, let us introduce the following symbols: r - risk function determined from pairs of sets ( A u A 2)\
L - loss function, function determining the loss we inflict, when taking the decision on classifying an observation from a sample;
Cj - event consisting in th at a given sample element comes from the first population;
C 2 - event consisting in th at a given sample element comes from the second population;
D j - event consisting in taking the decision th at a given sample element comes from the first population;
D2 - event consisting in taking the decision that a given sample element comes from the second population;
Let us notice that for each element of the sample X u ..., X„ the loss function L is described by the formula:
!
L x, when C t n D 2 L z, when C 2 n D x0, when (C t n D y) u (C 2 n Z)2)
Therefore, we can determine the expectation of the loss function, that is the risk function, in the following way:
r = E(L) = L ÍP ( C 1 n D2) + L 2P( C2 n D J =
= L i P ( C 1)P(D2/ C l + L 2P ( C2) P ( D J C 2), (2.2) hence
r = PiLl\ f i ( x ) d x + p 2L 2 \ f 2(x)dx (2.3)
Ат. A \
We consider risk as the function of the sets A y and A 2. Hence, we search for such sets of A t and A 2, that the function r ( A t , A 2) reached the least value. Let us notice
r (Au A J ^ V i L y $ f i ( x ) d x + PlLi ^ f i ( x ) d x + ^ \ p 2L 2f 2( x ) - p l L 1f 1(x)]dx = A j A \ A i = PiL i + Í [ P z W2W •- P i L J t (x)]dx (2.4) A i and r (Au A 2) — p2L 2 \ f 2(x)dx + p 2L 2 \ f 2(x)dx + j [ P j L / ^ x ) - p 2L / 2(x)]dx = = PiL 2 + Í [ P i L J x(x) - p 2L 2f 1(x)]dx (2.5) Aj
So we reach the following description o f optim um A t sets A 2: A t = { x e R : р 2Ь ^ 2{х) < p ^ L J ^ x ) }
A 2 = { x e R : p ^ L J ^ x ) ^ p 2L / 2(x)}
(2
.6
) (2.7) T he sign o f unstrict inequality occurs in form ula (2.6), and strict inequality in form ula (2.7). Because of the continuity o f the investigated populations, distributions it is insignificant. There could occur a reverse case, which would not lead to different results o f our considerations, i.e. to changes in the level of risk.So we showed that in case when we know the probability p l th at an observation comes from the first population, we are able to set optim um in terms o f minimizing the risk function, sets A 1 and A 2 allowing to take decision o f classifying a sample element to one of the possible populations.
The switching regression m odel is a particular case o f a statistical m odel with random coefficients. In this model, the coefficients can take only finite num ber of values. It m eans th at the explained variable can have distribution belonging to one o f several possible classes o f distributions, th at is, values o f the variable can be determined (generated) by one of several possible states o f setting. Some switching regression models are applied in the m arket disequilibrium analysis (see, e.g. F a i r and J a f f e e 1972, F a i r and K e l e j i a n 1974, H a r t l e y and M a i l e i a 1977, L a f f o n t and M o n f o r t 1979).
In this paper we shall deal with a particular case o f the form of a swit-ching regression model (see, e.g. Q u a n d t 1972, K i e f e r 1978, C h a r e m z a
1981, p. 94-87, T o m a s z e w i c z 1985, p . 442-446, P r u s k a 1987):
where t — 1, T and и T2 = {1, 2 ,..., T } and Tt n T 2 = 0 , when the sets o f indices Tv and T2 can be either know n or unknow n. O ther symbols are as follows:
y t - variable explained by the model;
x lt, x 2t - colum n vectors o f the explanatory variables:
a u a2 - column vectors of the m odel’s structural param eters;
e2t - random com ponents o f the model; random variables with norm al distributions with null expected values and variances a \ and a 2, respectively, such that
3. T H E SW IT C H IN G R EG R ESSIO N M O D EL
x 'u oci + F.u for t e T i
cov(elt, B2t) = 0, cov(elt, eu ) = 0, cov(e2t, e2t) = 0, cov(elt, e2t) = 0, for x and t, т е { 1 ... T).
T he m odel’s param eters (3.1) can be estimated by different m ethods. We m ay apply the Bayesian estim ation (see e.g. F e r r e i r a 1975, S w a m у and M e h t a 1975) or non-Bayesian estim ation (see e.g. F a i r and J a f f e e 1972, F a i r and K e l e j i a n 1974, Q u a n d t and R a m s e y 1978, S c h m i d t 1982). If we use the m aximum likelihood m ethod (M L-estim ation) we do it in two stages. The first stage consists of determining the likelihood function for a given model. The second stage is setting the point in which the function reaches its maximum. In this paper we shall only deal with the form o f the likelihood function depending on the inform ation we have on sets Tj and T2.
If we know sets x v and t 2 then the likelihood function for the m odel (3.1) is determ ined by the following form ula (see, e.g. G o l d f e l d and Q u a n d t 1972, p. 258-262, P r u s k a 1987, p. 21):
where = cardTu x2 = cardT2.
I f we do not know sets Tt and T2, then m odel (3.1) can be written down in the form:
_ f * ita i + e1( with probability p Y
^ (*2t“ 2 + £2r with probability p 2 = l —p 1
where 0 < Pj < 1 and p l can assume either know n or unknow n value. The likelihood function for the m odel (3.3.) is described by the for-m ula (see, e.g Q u a n d t 1972, C h a r e for-m z a 1981, p. 116, P r u s k a
1987, p. 24):
In the process o f estimating the param eters of the switching regression models, one can take advantage o f additional inform ation on the sample division. If we have at our disposal observations o f the variable dt for t = 1 , T, the distribution of which is in the form:
L(a
t , a 2, a\, a\) = (2rc) T'2a l f‘a \ tJexp j -~
J] (yt - x ' ^ J 21 teT i
(3.2)
P(dt = l ) = P ( t e T 1) = p 1
P{dt = 0) = P ( t e T 2) = p 2 W
then the proper use o f these d ata can result in increased efficiency of estim ators obtained by the likelihood m ethod (see K i e f e r 1978, 1979, L e e and P o r t e r 1984). Variable dt is sometimes called the sample division indicator.
In the case when observations of variable dt are available, for the M L estim ation of the m odel’s param eters (3.3) we can use the jo in t density function o f variables (yt, d j , which is described by the formula:
КУ„ d,) = d f & J d , = 1 )P(dt = 1) + (1 - dt) f 2(yt \dt = 0)P(dt = 0) (3.6) where f { and f 2 are conditional density functions o f variable y t, when, respectively, dt = 1 or dt = 0 (for the m odel (3.3), these are densities o f the norm al distributions, different in param eters). The likelihood function built on the form ula (3.6) assumes value:
г
U p » “ 2. ffi, Pi) = U A y * dt) (3.7)
t=i
where p L is either know n or unknow n value. If we know p x, we need not estim ate the param eter and then the function (3.7) depends only on a u «2» ° ь °2- T he m odel’s param eters estim ators (3.3) obtained in the process o f maximizing the likelihood function (3.7) are m ore efficient than the estim ators obtained from the function (3.4) (see K i e f e r 1979).
4. D IS C R IM IN A TIV E C O N ST R U C TIO N O F T H E SA M PLE D IV ISIO N IN D IC A T O R
From the considerations presented in the works by K i e f e r (1979) and L e e and P o r t e r (1984) it follows, that having extra inform ation on the observations classification, which is provided by the sample division indicator, results in increasing the efficiency o f the M L-estim ators of the switching regression models. There arises a question whether there is also a possibility to construct the sample division indicator. So far the observable variables (or their transform ations) linked to the examined process described by m eans o f the switching regression model, have been assumed as indicators. C onstructing the sample division indicator is suggested in the same way as there are created sets o f values o f an investigated feature in the discrim inant analysis, which quarantee m inimum risk while taking decisions on including the sample element to one o f the two possible populations. Some similarities between problem s appearing in the switching regression analysis and the
discrim inant analysis have already been noticed in the work by K i e f e r (1980). It includes a suggestion o f a new m ethod o f estimating the switching regression m odel’s param eters, alternative to the m axim um likelihood m ethod, but not to the construction of the sample division indicator. F o r, the m odel (3.1), wherein sets and T2 are not know n, let us create variable dt o f the form:
, fl dla t e T ,
■Ho
dla , e T , <4 I >and Lj and L 2 are values of the loss which is inflicted, when undertaking a wrong decision (i.e. assuming element y, determined by the first equation as generated by the second equation, or vice versa); f u and f 2t are densities o f random variable yv when it is determined by the first and second equation, respectively.
Variable dt described by the form ula (4.1) can play the p art of sample division indicator for the modej (3.3), when probability pj is know n and densities f u and f 2t are known, too. In case o f the switching regression models we usually lack such inform ation. Theorefore, the M L-estim ation o f these models, using the discriminative indicator o f sample division can be performed only after estimating the m odel’s param eters by the m axim um likelihood m ethod without an indicator. Then, the param eters of distributions determined by densities f lt and J 2t and probabilities p t , p 2 will also be estimated. Second estim ation of the m odel’s coefficients aims at increasing efficiency o f their estimators. One should also notice that determ ining sets A i t and A 2t allows to define for each t e T x KjT2 a group o f variables (factors), through which value y t was generated.
In the paper there has been suggested a construction of the sample distribution indicator for a m odel o f switching regression, using some elements of discrim ination analysis. D ue to this, the switching regression models can be used not only for describing and forecasting phenom ena
where
Tx = {0 < í < T: yt e A u }, T2 = {0 < t < T : y t e A 2t}
•^lr = {yE R- Pz^if 2i(ý) ^ PiLjf irOO}
^2t ~ {y^ R- P i ^ j f it(y)
^
rCv)}
Pi = P i t e T J , p 2 = P ( t e T 2) (4.2) (4.3) (4.4) (4.5) 5. F IN A L R EM A R K Sgenerated by various groups of factors, but also for determining which group o f factors set a given value o f the observed random variable. Furtherm ore, after reestim ating of the m odel’s param eters using the indicator of sample distribution, one can expect larger efficiency of estimators. To investigate the properties o f these estim ators we need relevant sim ulation experiments.
R EFE R E N C E S
C h a r e m z a W. (1981): Ekonometryczne modele nierównowagi. Problemy specyfikacji i estymacji, „Zeszyty N aukow e Uniwersytetu G dańskiego” , G dańsk.
F a i r R., J a f f e e D. (1972): Methods o f Estimation fo r M arkets in Disequilibrium, „Econom et- rica” , 40, p . 497-514.
F a i r R., K e l e j i a n H . (1974): M ethods o f Estimation fo r M arkets in Disequilibrium. A Further Study, „Econom etrica” , 42, p. 177-190.
F e r r e i r a P. (1975): A Bayesian Analysis o f a Switching Regression Model: Known Number o f Regimes, „Journal o f the A m erican Statistical A ssociation” , 70, p. 370-374.
G o l f e l d S., Q u a n d t R . (1972): Nonlinear M ethods in Econometrics, „N o rth -H o llan d Publishing Com pany” .
H a r t l e y M. and M a l l e l a P. (1977): The Asym ptotic Properties o f a M axim um Likelihood Estimator fo r a M odel o f M arkets in Disequilibrium, „Econom etrica” , 45, p. 1205-1220. K i e f e r N . (1978): Discrete Parameter Variation: Efficient Estimation o f a Switching Regression
Model, „Econom etrica” , 46, p. 427-434.
K i e f e r N . (1979): On the Value o f Sample Separation Information, „Econom etrica” , 47, p. 997-1003.
K i e f e r N. (1980): A Note on Switching Regressions and Logistic Discrimination, „Econom etrica”, 48, p. 1065-1069.
L a f f o n t J. and M o n f o r t A. (1979): Disequilibrium Econometrics in Dynamic Models, „Journal o f Econom etrics” , 11, p. 353-361.
L e e L. F., P o r t e r R. H. (1984): Switching Regression Models with Imperfect Sample Separation Information - with an Application on Cartel Stability, „E conom etrica” , 52, p. 391-418.
P r u s k a К . (1987): Zastosowanie m etody największej wiarygodności i regresji przełącznikowej do estymacji ekonometrycznych modeli nierównowagi, D o cto r’s dissertation, Łódź.
Q u a n d t R. (1972): A New Approach to Estimating Switching Regressions, „Journal o f the A m erican Statistical A ssociation” , 67, p. 306-310.
Q u a n d t R., R a m s e y J. (1978): Estimating M ixtures o f Normal Distributions and Switching Regressions, „Journal o f the A m erican Statistical A ssociation” , 73, p. 730-738.
S c h m i d t P. (1982): A n Improved Version o f the Quandt-Ram sey M G F Estimator fo r M ixtures o f Norm al Distributions and Switching Regressions, „E conom etrica” , 50, p. 501-516. S w a m y P., M e h t a J. (1975): Bayesian and Non-Bayesian Analysis o f Switching Regression
and o f Random Coefficient Regression Models, „Jo u rn al o f the A m erican Statistical A ssociation” , 70, p. 593-602.
T o m a s z e w i c z A . (1985): Jednorównaniowe modele ekonometryczne przy nieklasycznych założeniach, W U Ł, Łódź.
Z u b r z y c k i S. (1970): Wykłady z rachunku prawdopodobieństwa i sta tystyki matematycznej, PW N , W arszawa.
Krystyna Pruska
O K LA SY FIK A C JI O BSERW ACJI W R E G R ESJI PR ZE Ł Ą C Z N IK O W E J
W pracy zaproponow ana jest m etoda wyznaczania indykatora podziału próby d la pewnego modelu regresji przełącznikowej z dw om a stanami generującymi w artości zmiennej objaśnianej. Indykator ten zapewnia najmniejsze ryzyko popełn ienia pomyłki przy klasyfikacji obserwacji rozum iane ja k o w artość oczekiwana odpowiedniej funkcji straty. Przy konstrukcji tego indykatora wykorzystuje się elementy analizy dyskryminacji.
