Download Assignment 3, Machine Learning - Artificial Intelligence | COSC 6368 and more Assignments Computer Science in PDF only on Docsity!
COSC 6368 (Fall 2009)
Assignment 3: Machine Learning
- Decision Tree Induction w= a) We would like to predict the gender of a person based on two binary attributes: leg-cover (pants or skirts) and beard (beard or bare-faced). We assume we have a data set of 20000 individuals, 12000 of which are male and 8000 of which are female. 80% of the 12000 males are barefaced. Skirts are present on 50% of the females. All females are bare-faced and no male wears a skirt. [3]
i) Compute the information gain of using the attribute leg-cover for predicting gender!
Gain(leg-cover) = H(3/5, 2/5) - 4/5 H(3/4, 1/4) – 1/5H(0, 1) = H(3/5, 2/5) – 0. 649
ii) What is the information gain of using the attribute beard to predict gender?
Gain(Beard) = H(3/5, 2/5) - 22/25 H(6/11, 5/11) – 3/25H(1, 0) = H(3/5, 2/5) – 0. 875
iii) Based on yours answers to i) and ii) which attribute should be used as the root of a decision tree?
Because Gain(leg-cover) > Gain(Beard) => Using leg-cover as the root of decision tree
iv) Neural Networks w= Assume we have the perceptron that is depicted in Fig. 1 that has two regular inputs, X1 and X2, and an extra fixed input X3, which always has the value 1. The perceptron's output is given as the function:
Out= If (w1X1 + w2X2 + w3*X3) > 0 then 1 else 0
Note that using the extra input, X3, we can achieve the same effect as changing the perceptron's threshold by changing w3. Thus, we can use the same simple perceptron learning rule presented in our textbook to control this threshold as well.
A. We want to teach the perceptron to recognize the function X1 XOR X2 with the following training set:
X1 X2 X3 Out
1 1 1 0 0 1 1 1 1 0 1 1 0 0 1 0
Show the change in the weights of the perceptron for every presentation of a training instance. Assume the initial weights are: w1=0.1, w2=0.1, w3=0.9 Important: Do the iterations according to the order of the samples in the training set. When you finish the four samples go over them again. You should stop the iterations once you get convergence, or when the values you get indicate that there will be no convergence. In either case explain your decision to stop the iterations. Assume in your computations that the learning rate α is 0.4.
B. This time, instead of being limited to a single perceptron, we will introduce hidden units and use a different activation function. Our new network is depicted in Fig. 2. Assume that the initial weights are w14 = 0.1, w15 = 0.1, w24 = 0.1, w25 = 0.1, w34 = 0.1, w35 = 0.1, w36 = 0.2, w46 = 0.2, and w56 = 0.2. The training set is the same as in (A). Use γ=0.3 as your learning rate. Show what the new weights would be after using the backpropagation algorithm for two updates using just the first two training instances. Use g(x) = 1/(1+e(-x)) as the activation function; that is g'(x)=(e(-x))/(1+e(-x))2).
S# X1 X2 X3 Out True_Out Error w14 w15 w24 w25 w34 w35 w36 w46 w
0 0.1 0.1 0.1 0.1 0.1 0.1 0.2 0.2 0. 1 1 1 1 0 2 0 1 1 1
[2.5]
First Update: Initial output: H4 = g(w14(X1) + w24(X2) + w34(X3)) = g(0.1(1) + 0.1(1) + 0.1(1)) = g(0.3) =1/(1+e(-0.3)) = 0. H5 = g(w15(X1) + w25(X2) + w35(X3)) = g(0.1(1) + 0.1(1) + 0.1(1))=g(0.3) =1/(1+e(-0.3)) = 0. O6 = g(w46(H4)+w56(H5)+w36(X3)) = g(0.2(0.57) + 0.2(0.57) + 0.2(1)) = g(0.429777) ==1/(1+e**(-0.429777)) = 0.
Errors: delta6 = (T-O6)O6(1-O6) = (0-0.60582)* 0.60582(1-0.60582) = -0. delta5 = delta6w56H5(1-H5) = -0.144670.20.574443(1-0.574443) = -0. delta4 = delta6w46H4(1-H4) = -0.144670.20.574443*(1-0.574443) = -0.
Adjusted weights: w14 = w14 + γ∗X1delta4 = 0.1 + 0.31*(-0.00707) = 0.
w15 = w15 + γ∗X1delta5 = 0.1 + 0.31(-0.00707) = 0. w24 = w24 + γX2delta4 = 0.1 + 0.31(-0.00707) = 0. w25 = w25 + γX2delta5 = 0.1 + 0.31(-0.00707) = 0. w34 = w34 + γX3delta4 = 0.1 + 0.31(-0.00707) = 0. w35 = w35 + γX3delta5 = 0.1 + 0.31(-0.00707) = 0. w36 = w36 + γX3delta6 = 0.2 + 0.31(-0.14467) = 0. w46 = w46 + γH4delta6 = 0.2 + 0.30.574443(-0.14467) = 0. w56 = w56 + γH5delta6 = 0.2 + 0.30.574443*(-0.14467) = 0.
Second Update:
New output: H4 = g(w14(X1) + w24(X2) + w34(X3)) = g(0.097878 (0) + 0.097878 (1) + 0.097878 (1)) = g(0.195756) =1/(1+e(-0.195756)) = 0. H5 = g(w15(X1) + w25(X2) + w35(X3)) = g(0.097878 (0) + 0.097878 (1) + 0.097878 (1)) = g(0.195756) =1/(1+e(-0.195756)) = 0. O6 = g(w46(H4)+w56(H5)+w36(X3)) = g(0.175068 (0.548783) + 0.175068 (0.548783) + 0.156599 (1)) = g(0.348748) =1/(1+e**(-0.348748)) = 0.
Error: delta6 = (T-0)O6(1-O6) = (1-0.586314)0.586314(1-0.586314) = 0. delta5 = delta6w56H5(1-H5) = 0.100340.1750680.548783(1-0.548783) = 0. delta4 = delta6w46H4(1-H4) = 0.100340.1750680.548783(1-0.548783) = 0.
Adjusted weights: w14 = w14 + γ∗X1delta4 = 0.097878+ 0.30(0.00435) = 0. w15 = w15 + γ∗X1delta5 = 0.097878+ 0.30(0.00435) = 0. w24 = w24 + γX2delta4 = 0.097878+ 0.31(0.00435) = 0. w25 = w25 + γX2delta5 = 0.097878+ 0.31(0.00435) = 0. w34 = w34 + γX3delta4 = 0.097878+ 0.31(0.00435) = 0. w35 = w35 + γX3delta5 = 0.097878+ 0.31(0.00435) = 0. w36 = w36 + γX3delta6 = 0.156599+ 0.31(0.10034) = 0. w46 = w46 + γH4delta6 = 0.175068+ 0.30.548783(0.10034) = 0. w56 = w56 + γH5delta6 = 0.175068+ 0.30.548783(0.10034) = 0.
S# X1 X2 X3 Out True Out
Error w14 w15 w24 w25 w34 w35 w36 w46 w
of the first part of the experiment for question b with the results you obtained for question a.
[3]
Using Temporal Difference Learning
UΠ^ (s) UΠ^ (s) + α [ R(s) + γ UΠ^ (s’) - UΠ^ (s) ]
For this problem the policy is selected randomly. No fix result was given. We can run algorithm for several times and get the average values.
Below is one example result:
Run State 1 State 2 State 3 State 4 State 5 State 6 State 7 State 8 State 9 State 10
1 4.233 5.247 11.355 2.769 5.46 7.02 -3.215 5.14 10.038 3.
AVG 4.51 5.345 11.272 2.403 5.513 7.004 -2.765 5.612 10.02 4.
FLIPPING REWARDS
AVG 0.709 -1.179 -7.708 0.625 -3.536 -6.993 3.448 -6.176 -10.01 -4.
When we reverse the rewards associated with states (positive rewards become negative, and negative rewards become positive), it means that the STU world changes and we have to show that the algorithm can able to learn and adapt utility estimations. For example, State 3 and 9 which had the highest utility values in the first loop became the worst after flipping its reward value. And the state 7 that had the worst value basically because of its negative reward became the most attractive after flipping the rewards.
Comparison: In order to compare the two methods, we will rank the states based on the average utilities from the results
Rank #1 #2 #3 #4 #5 #6 #7 #8 #9 # Bellman 3 9 1 5 2 6 8 4 10 7 TD 3 9 6 8 5 2 1 10 4 7
c) Based on the findings you obtained in steps a and b devise a good policy for an agent to navigate through the STU world!! Write a 2-3 page report that summarizes the findings of the project. Be aware of the fact that at least 30% of the available points are allocated to the interpretation of the experimental results.
