Lectures 5,6
MACHINE LEARNING
EXPERT SYSTEMS
Contents
Machine learning
Knowledge representation
Expert systems
INDUCTION OF DECISION
TREES FROM DATA
Sunny Rain Overcast
Outlook
Humidity Wind
High Normal Strong Weak
Outlook Temp. Humid. Wind Sport?
1 Sunny Hot High Weak No
2 Sunny Hot High Strong No 3 Overcast Hot High Weak Yes 4 Rain Mild High Weak Yes 5 Rain Cold Normal Weak Yes 6 Rain Cold Normal Strong No 7 Overcast Cold Normal Strong Yes 8 Sunny Mild High Weak No 9 Sunny Cold Normal Weak Yes 10 Rain Mild Normal Weak Yes 11 Sunny Mild Normal Strong Yes 12 Overcast Mild High Strong Yes 13 Overcast Hot Normal Weak Yes
Decision trees
Sport=No Sport=Yes
Data from credit history of loan applications
A simplified tree…
But how to do it?
The induction algorithm ID3
Partially constructed decision trees
STEP 1
STEP 2
A heuristic problem
HOW TO SELECT THE BEST PROPERTY?
Approximate trees
High Normal
Humidity
85%
Outlook
75%
100%
Sunny Not Sunny
Outlook Temp. Humid. Wind Sport?
1 Sunny Hot High Weak No
2 Sunny Hot High Strong No 3 Overcast Hot High Weak Yes 4 Rain Mild High Weak Yes 5 Rain Cold Normal Weak Yes 6 Rain Cold Normal Strong No 7 Overcast Cold Normal Strong Yes 8 Sunny Mild High Weak No 9 Sunny Cold Normal Weak Yes 10 Rain Mild Normal Weak Yes 11 Sunny Mild Normal Strong Yes 12 Overcast Mild High Strong Yes 13 Overcast Hot Normal Weak Yes
CLASSIFICATION
SYSTEMS
A full classification system
Pattern recognition
Patterns:
– images, personal records, driving habits, etc.
Representation:
– vector of features (inputs to a neural network)
Pattern classification:
– Classify a pattern to one of the given classes
> classifier < Marks
> classifier < not Marks
> classifier < not Marks
> classifier < not Marks
> classifier < Marks
> classifier < not Marks
Classifier training
Classifier application
> Classifier > Marks
Note: The test image does not
appear in the training data
LEARNING IN GENERAL
The data and the goals
We begin with a collection of positive (and usually negative) examples of a target class (a concept to be learnt)
The goal is to infer a general definition
that will allow the learner to recognize
future instances of the class
Knowledge representation
Positive and negative examples can be represented, e.g., in predicate calculus
Two positive instances of the concept of
“ball” can be expressed as follows:
size(obj1,small)
color(obj1,red)
shape(obj1,round) size(obj2,large)
color(obj2,red)
shape(obj2,round)
The general concept of “ball” could be defined by:
size(X,Y)
color(X,Z)
shape(X,round)
where any sentence that unifies with
this general definition represents a ball
A general model of the learning
process
A set of operations
Given a set of
training instances, the learner must construct a
generalization,
heuristic rule or
plan that satisfies
The concept space
Representation language and the operations define a space of
potential concept definitions
The learner must search this space to find the
desired concept
Heuristic search
Learning
programs must commit to a direction and order of search, as well as…
…to the use of available data and heuristics to
search efficiently
PATRICK WINSTON’S PROGRAM ON
LEARNING CONCEPTS
Examples and near misses for
the concept “arch”
Generalization of descriptions to
include multiple examples (I)
Generalization of descriptions to
include multiple examples (II)
Specialization of a description to exclude a
near miss
so that this can’t match Starting with the original
we add special
constraints
A BRIEF HISTORY OF AI REPRESENTATIONAL
SCHEMES
Semantic network developed by
Collins & Quillian
in their research on
human information
storage and
response times
Network
representation of properties of…
…snow and ice
…three definitions of the word “plant”
Three planes
representing…
Intersection path between “cry”
and “comfort” (Quillian 1967)
Case frame representation of
the sentence “Sarah fixed the
chair with glue.”
Conceptual dependency theory of four primitive conceptualizations
For example, all actions are assumed to reduce to
one or more of the primitive ACTs listed below:
“John ate the egg”
“John prevented Mary from giving a book to
Bill”
Restaurant
script
Restaurant script
(continued)
FRAMES
A frame includes:
Frame identification information
Its relationship to other frames
Descriptors of requirements
Procedural information on use of the structure described
Frame default information
New instance information
Relationship to other frames
For instance, the “hotel phone”
might be a special instance of
“phone”, which might be an
instance of a “communication
device”
Descriptors of requirements
For instance, a chair has its seat
between 20 and 40 cm from the floor, its back higher than 60 cm, etc.
These requirements may be used to
determine when new objects fir the
stereotype defined by the frame
Procedural information
An important feature of frames
is the ability to attach procedural
code to a slot
Frame default information
These are slot values that are taken to be true when no evidence to the contrary has been found
For instance, chairs have four legs,
telephones are pushbutton, hotel
beds are made by the staff
New instance information
Many frame slots may be left
unspecified until given a value for a particular instance or when they are needed for some aspect of problem solving
For instance, the color of the
bedspread may be left unspecified
Part of a frame description of a hotel room
“Specialization” indicates
a pointer to a superclass
Spatial frame for viewing a cube
(Minsky 1975)
CONCEPTUAL GRAPHS:
A NETWORK LANGUAGE
Conceptual
relations of
different arities
“Mary gave John the book”
“A dog named
emma is brown”
Examples of restriction…
…join, and simplify operations
Inheritance in conceptual graphs
“Tom believes that Jane likes pizza”
This example shows the use
of a propositional concept
RULE BASED
EXPERT SYSTEMS
Architecture of a typical expert system for a particular problem
domain
Guidelines to determine whether a problem is appropriate for
expert system solution (1)
The need for the solution justifies the cost and effort of building an expert system
Human expertise is not available in all situations where it is needed
The problem may be solved using
symbolic reasoning
Guidelines to determine whether a problem is appropriate for
expert system solution (2)
The problem domain is well structured and does not require commonsense reasoning
The problem may not be solved using traditional computing methods
Cooperative and articulate experts exist
The problem is of proper size and scope
Reasoning with
a typical expert
system
The role of mental or conceptual
models in problem solving
A small
system for
analysis of
automotive
problems
The and/or graph
searched in the car
diagnosis example
The production system at the
start of a consultation in the car diagnostic example
Imagine that we want to get information about
spark plugs
The production system after
Rule 1 has fired
The system after Rule 4 has fired
Note the stack-based approach to goal reduction
