Download Support Vector Machines and Classification Trees Homework Solution - Prof. Feng Liang and more Assignments Statistics in PDF only on Docsity!
STAT 542: HOMEWORK 3
1. SUPPORT VECTOR MACHINE
For linear kernel:
(b) Among the values { 0. 01 , 0. 1 , 1 , 10 , 100 }, the optimal parameter cost cho- sen by 5-fold CV is 0.1. The corresponding CV error is 0.127 and the number of support vectors is 199. (c) The classification results on the test data are shown in the following table: fˆ (x) = 0 fˆ (x) = 1 Y=0 226 16 Y=1 25 133 and the misclassification rate is 0.103.
For Gaussian kernel:
(b) Among the values { 0. 01 , 0. 1 , 1 , 10 , 100 }, the optimal parameter cost cho- sen by 5-fold CV is 10. The corresponding CV error is 0.115 and the number of support vectors is 245. (c) The classification results on the test data are shown in the following table: fˆ (x) = 0 fˆ (x) = 1 Y=0 231 11 Y=1 18 140 and the misclassification rate is 0.073.
- CLASSIFICATION TREE
(a) Construct the classification as shown in Figure 1, and the prediction on the test data is:
Wremove < 0.015|
Cdollar < 0.
Wfree < 0. CAPlongest < 8. Cexclam < 1.739 Whp < 0. Wre < 0.285Wbusiness < 0. Wgeorge < 0.
Cexclam < 0.
Whp < 0. CAPave < 1. Wwill < 0.
Wremove < 2.
0 1 0 0 0
1 0
(^0 1 0 1 )
0
1 1
Figure 1: Classification tree constructed by the training data.
fˆ (x) = 0 fˆ (x) = 1 Y=0 235 7 Y=1 27 131 Print out the tree using ”print.tree”, we have: 33)Cexclam > 1. 73968 .3181(0. 5000000 .500000)∗ that meas it has exactly half-half chance to predict 1 or 0 in the leaf ”Cexclam”. So the decision rule is just random guessing.
(b) The pruning result is shown in left of Figure 2, and the optimal sub- tree with cost-complexity measure equal to 15 (i.e., k=15) is depicted in Figure 2 (right). (c) Using 5-fold CV, we choose the tree with size 11, shown in Figure 3.
(d) Here, we use the deviance as the impurity measure to prune the tree. Construct the tree with the optimal size 11 (Figure 4) and the classifica- tion results on the test data is fˆ (x) = 0 fˆ (x) = 1 Y=0 235 7 Y=1 26 132
Wremove < 0.015|
Cdollar < 0.
Wfree < 0. CAPlongest < 8. Cexclam < 1.739 Whp < 0. Wre < 0.285Wbusiness < 0. Wgeorge < 0.
Cexclam < 0.
Whp < 0. CAPave < 1. Wwill < 0.
Wremove < 2.
0 1 0 0 0
1 0
(^0 1 0 1 )
0
1 1
Figure 4: The classification tree constructed with size 11.
which implies the misclassification rate is 0.083.
(e) Fit the training data using randomF orest. The classification results on the test data is: fˆ (x) = 0 fˆ (x) = 1 Y=0 237 5 Y=1 16 142 that means the misclassification rate is 0.053.
