Neural network- laboratory
Agnieszka Nowak - Brzezińska
Neural network structure
1. Choose randomly one of the observation
2. Go through the appropriate procedures
to determine the input value
3. Compare the desired value with
the actually obtained in the
network
4. Adjust the weight by calculating the error
Learning process
How to calculate the prediction error ?
where:
•Errori is the error from the i-th node,
•Outputi is the value predicted by a network,
•Actuali is the real value (which the network should learn).
Change the weights
L- is so called learning network ratio. (usually values are from [0,1]). The less is the value of this coefficient the slower the learning process is.
Often this ratio is set to the highest value initially, and then is reduced by re-weighting network.
Example
Example
The network example
Wejście 1 Wejście 2
Bias Wejście 0
sumator
Funkcja aktywacji Wyjście
Waga 1 Waga 2
Waga 0
Sumator = wejście 0 * waga 0 + wejście 1 * waga 1 + wejście 2 * waga 2
0 0 1
sumator
Funkcja aktywacji 0,3 >0
0,5 -0,4
0,3
Sumator = 1*0,3 + 0 * 0,5 + 0 * -0,4 = 0,3
1
0 1 1
sumator
Funkcja aktywacji -0,2 <0
0,5 -0,4
0,2
Sumator = 1*0,2 + 0 * 0,5 +1 * -0,4 = -0,2
0
Create the sheet
The Algorithm
1. calculate the sum of the inputs with weights 2. The value put to the activation function
3. Calculate the output value
4. If output value differs from the pattern calculate the prediction error and change the weights
5. Repeat steps 1-4 as long as the output value differs from the patterns that it has to learn.
The STOP condition…
• Calculate the sum of errors in each iteration
• If the error >0 the network still should learn…
HOMEWORK
excel / Calc / Gnumeric
Create the sheet of learning neural network which has 2 input nodes and 1 output node:
1. AND 2. OR 3. NOR 4. NAND 5. XOR
Logical operation
Answer for questions:
• In how many iterations the network has already learnt?
• Is there some kind of logical function which the network could not learn?
• If yes, do you know WHY ?
