Consider learning the logical ``or'' function.
We'll use a single perceptron with three inputs (x_1, x_2, x_3) and
single output ($l$). Note that the input x_3 is an artificial input
(i.e. the bias node) will remain fixed at -1.
Our examples are:
example x_1 x_2 x_3 l
====================================
1 0 0 -1 0
2 0 1 -1 1
3 1 0 -1 1
4 1 1 -1 1
We'll set the learning rate to 0.5 and initialize the weights to 0's:
=<0,0,0>
Epoch One
=========
Example 1 <0,0,-1>:
dot product of example and weights is <0,0,-1> x <0,0,0> = 0
since 0 >= threshold of 0, output = 1
desired output = 0
error = -1
so update weights:
<0,0,0> + <.5 (learning rate) x 0 (x_1) x -1 (error),
.5 x 0 x -1,
.5 x -1 x -1> = <0,0,0> + <0,0,.5> = <0,0,.5>
=<0,0,.5>
Example 2 <0,1,-1>:
dot product of example and weights is <0,1,-1> x <0,0,.5> = -.5
since -.5 < threshold of 0, output = 0
desired output = 1
error = 1
so update weights:
<0,0,.5> + <.5 x 0 x 1, .5 x 1 x 1, .5 x -1 x 1> =
<0,0,.5> + <0,.5,-.5> = <0,.5,0>
=<0,.5,0>
Example 3 <1,0,-1>:
dot product of example and weights is <1,0,-1> x <0,.5,0> = 0
since 0 >= threshold of 0, output = 1
desired output = 1
error = 0
example is correct, so no weight updates.
=<0,.5,0>
Example 4 <1,1,-1>:
dot product of example and weights is <1,1,-1> x <0,.5,0> = .5
since .5 >= threshold of 0, output = 1
desired output = 1
error = 0
example is correct, so no weight updates.
=<0,.5,0>
Epoch Two
=========
Example 1 <0,0,-1>:
dot product of example and weights is <0,0,-1> x <0,.5,0> = 0
since 0 >= threshold of 0, output = 1
desired output = 0
error = -1
so update weights:
<0,.5,0> + <.5 x 0 x -1, .5 x 0 x -1, .5 x -1 x -1> =
<0,.5,0> + <0,0,.5> = <0,.5,.5>
=<0,.5,.5>
Example 2 <0,1,-1>:
dot product of example and weights is <0,1,-1> x <0,.5,.5> = 0
since 0 >= threshold of 0, output = 1
desired output = 1
error = 0
example is correct, so no weight updates.
=<0,.5,.5>
Example 3 <1,0,-1>:
dot product of example and weights is <1,0,-1> x <0,.5,.5> = -.5
since -.5 < threshold of 0, output = 0
desired output = 1
error = 1
so update weights:
<0,.5,.5> + <.5 x 1 x 1, .5 x 0 x 1, .5 x -1 x 1> =
<0,.5,.5> + <.5,0,-.5> = <.5,.5,0>
=<.5,.5,0>
Example 4 <1,1,-1>:
dot product of example and weights is <1,1,-1> x <.5,.5,0> = 1
since 1 >= threshold of 0, output = 1
desired output = 1
error = 0
example is correct, so no weight updates.
=<.5,.5,0>
Epoch Three
===========
Example 1 <0,0,-1>:
dot product of example and weights is <0,0,-1> x <.5,.5,0> = 0
since 0 >= threshold of 0, output = 1
desired output = 0
error = -1
so update weights:
<.5,.5,0> + <.5 x 0 x -1, .5 x 0 x -1, .5 x -1 x -1> =
<.5,.5,0> + <0,0,.5> = <.5,.5,.5>
=<.5,.5,.5>
Example 2 <0,1,-1>:
dot product of example and weights is <0,1,-1> x <.5,.5,.5> = 0
since 0 >= threshold of 0, output = 1
desired output = 1
error = 0
example is correct, so no weight updates.
=<.5,.5,.5>
Example 3 <1,0,-1>:
dot product of example and weights is <1,0,-1> x <.5,.5,.5> = 0
since 0 >= threshold of 0, output = 1
desired output = 1
error = 0
example is correct, so no weight updates.
=<.5,.5,.5>
Example 4 <1,1,-1>:
dot product of example and weights is <1,1,-1> x <.5,.5,.5> = .5
since .5 >= threshold of 0, output = 1
desired output = 1
error = 0
example is correct, so no weight updates.
=<.5,.5,.5>
Epoch Four
==========
Every example is correct.
Example 1 <0,0,-1>:
dot product of example and weights is <0,0,-1> x <.5,.5,.5> = -.5
since -.5 < threshold of 0, output = 0
desired output = 0
example is correct, so no weight updates.
=<.5,.5,.5>
Example 2 <0,1,-1>:
dot product of example and weights is <0,1,-1> x <.5,.5,.5> = 0
since 0 >= threshold of 0, output = 1
desired output = 1
example is correct, so no weight updates.
=<.5,.5,.5>
Example 3 <1,0,-1>:
dot product of example and weights is <1,0,-1> x <.5,.5,.5> = 0
since 0 >= threshold of 0, output = 1
desired output = 1
example is correct, so no weight updates.
=<.5,.5,.5>
Example 4 <1,1,-1>:
dot product of example and weights is <1,1,-1> x <.5,.5,.5> = .5
since .5 >= threshold of 0, output = 1
desired output = 1
example is correct, so no weight updates.
=<.5,.5,.5>