A C T A U N I V E R S I T A T I S L O D Z I E N S I S FO L IA O EC O N O M IC A 141, 1997
Czesław Domański* Dorota Pekasiewicz*
R E M A R K S ON SOM E L E A R N IN G A L G O R IT H M S
Abstract. In this paper we present some problems co ncerning artificial intelligence.
In the first two points there are presented algorithm ic and heuristic procedures, which are applied in solving problem s and m aking optimal decision for the fixed states as follows: beginning state, set o f all states o f a problem, subset o f final states.
The fundam ental p a rt of the paper discusses three leam ing-algorithm s: ID 3, A Q and backpropagation, owing to which com puter basing on a given sample is to create a general form ula or rule and to make a decision which is a solution o f the problem .
The last section o f the paper includes a short description o f the predictive accuracy o f the algorithms.
Key words: algorithmic procedure, heuristic procedure, learning algorithms, backpropagation algorithm s.
1. P R EL IM IN A R IE S
Artificial intelligence is a notion, which has its origins in fantastic literature. M ore than 20 years ago it became a scientific term and began expressing something real, although difficult to define.
Intelligence is the capacity o f reasoning and perceiving and using possesed knowledge for solving new theoretical and practical problem s.
I he notion o f artificial intelligence relates to computers. C om puter, like a hum an being, can think, count and create, but these are results of a program m ade by m an. G enerally, artificial intelligence is building com puter program s, owing to which com puters behave in a way th at can be described as intelligent behaviour. C om puter can think about everything. It is applied to alm ost all dom ains, from general application in industrial work, through banking and economic operation, through application in
the army, politics, medicine to narrow specializations, for example verification o f astronom ical hypotheses.
C om puter surprises us with not only its speed, capacity o f faultless m em ory, m em ory size and scope of systematic research, that it is capable to carry out to reach its aim but also with the possibility of self-educating.
How does com puter think, how does it solve problems, w hat m ethods does it apply?
Firstly, algorithm ic methods. Algorithm is a set of actions, which when executed lead to solving problems. It is then a recipe for success. It is only really im portant, that it can be applied to m any different input data.
But solution o f a problem is often a choice of one alternative out of m any alternatives given by the environm ent of the problem . Decision is very difficult because the num ber of alternatives is often large and we m ust predict their consequences before the choice is m ade. Here, com puters work cannot be mechanical, which is typical for algorithm ic m ethods, but it m ust characterize with some intuition. Intuition is a heuristic m ethod o f solving problems. The idea of this m ethod is to reduce the size o f the tree of a problem by liquidation of some parts of configuration by ‘intelligent guess’, that they have small m eaning for solving the problem , or to simplify a complicated task and m ake the m ost of the results o f this simplification in solving the care problem or in dividing the problem into subproblem s and solving them successively. Heuristic proceeding saves time, but in opposition to algorithm ic it doesn’t guarantee a solution and in the case when we aquire it we d on’t known certainly if it is the best. This m ethod is also applied in the cases when we d o n ’t know any sensible algorithm of solving a problem.
2. FO R M U L A T IO N O F T H E PROBLEM
Heuristic program can be written as the following four: ( £ 0. £< 0
E 0 - beginning state,
£ - set o f all states of a problem, evidently E 0 e £,
£ - subset o f final states, which we w ant to obtain, SF - set o f operators, which are m apping 8
The solution o f the problem is the operator:
F — F l °Fz° ... оF„;
where F u F2, ..., F„ are operators, which lead to transition from the first state to the second, from the second to the third, etc.
Obviously F (E 0) = F„° ... оF t (E0) e £
and V Ft = 2F, E 0 c V f
i = l ...и i = l ...n
We can notice, th at Fi are defined only on subsets o f £.
I n heuristic m ethods com puter m ust m ake a decision alone. Therefore,
there m ust be a self-educating process performed by algorithms supplementing insufficient prior knowledge.
The basic learning system is given in the following figure:
Ideally the knowledge aquisition process should allow each expert to build his own expert system. These individual knowledge bases could then be combined into a representative knowledge base.
Inductive algorithm s are used for knowledge aquisition. These algorithm s are applied in constructing projects for assessing and predicting bankruptcies.
3. A SU RV EY IN D U C T IV E - L E A R N IN G A L G O R IT H M S
Three inductive - learning algorithm s will be presented: ID 3, A Q and backpropagation.
A lgorithm ID 3 has been widely studied and adopted in m any o f comm ercial packages. T he input to ID 3 is a collection o f exam ples, described by a vector o f m easurable values. The o utput is a decision tree with each branch corresponding to predicate (A t = VtJ), where Vu is the J-th value of attribute A t. In the case o f numeric attribute, value A t is split
U nknown __X Environm ent “ student” Decision M arker « --- ---Learning Section Ж Performance Evaluator ______ ^T eacher” f “ Learning System”
into two sets, forming two predicates (At > Vt) and (At < F,). The procedure to construct a classification tree involves splitting repeatedly a given set into disjoint, descendant subsets. Starting from root, which contains the entire training sample, an attribute is selected as splitting attribute at each level. Examples having identical values o f selected attribute are grouped into the same subset. This procedure is performed recursively for each subset until one of the following conditions is satisfied:
1) the subset is homogeneous,
2) there is no improvement in further splitting as indicated by chi-square test.
Exam ple o f a decision tree:
class 2 class 1 class 2 class 1 class 2 class 1
Decision tree represents a m ultistage decision problem . It describes possible outcomes and probabilities at each stage.
The likelihood of the final outcom e is the product o f the probabilities along the path representing this outcom e, from the root to the leaves. The tree can be represented by a set of rectangular regions as shown in the figure:
3
2
I ---
Algorithm AQ, in spite o f identical application as ID 3, differs from ID 3 in the language used. It is an extended version of PC (Predicate Calculus) named A PC (A nnotated Predicate Calculus). In this algorithm decision m aking instructions can include variables.
Example:
In ID3: If son_of_m ajor..client(yes>, then grant J o a n (y e s) F ather (X,Y) and m ajor _client(X) => grant _loan(Y )
T he third algorithm is connected with neural netw ork, exactly with networks called “feedforward”. Neural network consists o f set of homogeneous processing units connected in a direct graph. N eural netw ork represents knowledge as a pattern o f connections o f its proccesing units. Each example is described by a pair real vectors: input and output. A typical learning algorithm will be searching in the vector space for the weights best fitted to given example.
The backpropagation algorithm consists o f two phases: fo rw ard-propa- gation and backward-propagation. In the first phase an example is considered as a few input units. Messages are forwarded up the netw ork and output vector created by output units. Value Y, is com pared with the actual or desired output D t by calculating ( Y u - Du)z for 1 n where n is the
total num ber o f o utput units. The function error is defined as
E ±
I j z
The aim o f the algorithm is to find a set o f weights, that all input vectors are correctly m apped to their corresponding o utput vectors. Using function error E, the learning process can be described as the m inim ization o f function E in the weight space.
4. R E M A R K S A BO UT A PPLICA TIO N A N D PR ED IC TIV E A C C U R A C Y O F T H E A LG O R IT H M
The perform ance of the inductive-learning m odels is typically m easured by its predictive accuracy in a specific task dom ain. The m odel needs to be interpretable and m ust produce an accurate outcome. Obviously the accuracy of the decisions supported by models is m easured with the num ber o f correct classification, which are compared with real decision. Unfortunately, the predictive accuracy o f these algorithm s varies widely. F o r example, H y u n g - M i n and K a r Y a n (1992) employed the ID 3, A Q and back-propagation algorithm s in solving construction project perform ance tasks
and bankruptcy prediction tasks. As follows from their results, the accuracy o f decisions, which were m ade by m eans o f ID 3 or A Q algorithm s, fluctuates between 38-73% in construction project perform ance tasks and between 51-80% in bankruptcy prediction tasks.
The backpropagation performs rather consistenly. The predictive accuracy, unlike with other algorithm s, is in the range 58-73% for construction project perform ance assesment and in the range 85-89% for bankruptcy prediction.
The backpropagation stands out am ong the three algorithm s as having a better predictive accuracy.
R EFE R E N C E S
C a r g e Ch. ( 1981): Elementy informatyki. Inform atyka a m yśl ludzka, PW N , W arszawa. H o ł y ń s k i N. M . (1979): Sztuczna inteligencja. W arszawa.
H y u n g - M i n Michael Chung, K a r Y a n T am (1992): A comparative Analysis o f induc-
tive-learning algorithms, International Journal Intelligent Systems in A ccounting, Finance
and M anagem ent, p. 3-18.
D u b o i s D. , P r a d e H. , Y a g e r R. R. (1993): Fuzzy Sets fo r Intelligent Systems, M organ K aufm ann Publishers, Inc. Printed in USA, p. 888-900.
Czeslaw Domański, Dorota Pekasiewicz
U W AG I NA TEM A T K O M PU T ER O W Y C H A LG O R Y TM Ó W U C Z E N IA SIĘ
Praca dotyczy zagadnień sztucznej inteligencji. W pierwszych dwóch punktach przedstaw ione są procedury algorytm iczna i heurystyczna stosowane do rozwiązywania problem ów i p odjęcia optym alnej decyzji przy danych: stanie początkowym , zbiorze możliwych stanów decyzyjnych, podzbiorze ostatecznych stanów, które należy osiągnąć.
Z asadniczą część pracy stanowi omówienie trzech algorytm ów „uczenia się” : ID 3, A Q i „backpropagation” . Dzięki tym algorytm om kom puter na podstawie danej próby potrafi stworzyć wzór bądź regułę i podjąć decyzję będącą rozwiązaniem postawionego problem u.
