Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Material Type: Assignment; Professor: Roth; Class: Machine Learning; Subject: Computer Science; University: University of Illinois - Urbana-Champaign; Term: Fall 2008;
Typology: Assignments
1 / 3
This page cannot be seen from the preview
Don't miss anything!
CS446: Pattern Recognition and Machine Learning Fall 2008
Solution Handed In: September 25, 2008
a. ¬A ∨ B ∧ C [3 points]
b. (A ∧ ¬B) ∨ ¬(C ∧ D) [3 points]
c. (A ∨ B) ⊕ C ∨ A ⊕ (¬B ∧ C)[4 points]
Note that although the logical formula is complicated, it only involves three vari- ables so necessarily the tree must be at most depth four. If it was more than that then a variable would appear twice in some path from root, which is unnecessary.
a. The smallest possible decision tree consistent with a hypothesis in Dk contains 2 k + 1 nodes. Every non-leaf node has a branch labeled “1”/“0” (depending on whether xi or ¬xi is in the disjunction, respectively) leading to a leaf node labeled “1” and a branch labeled “0”/“1” leading to another non-leaf node, except for the lowest non-leaf node, that has a branch leading to a leaf node labeled “0”. b. The smallest possible decision tree consistent with a hypothesis in Ck contains 2 k + 1 nodes. Every non-leaf node has a branch labeled “0”/“1” (depending on whether xi or ¬xi is in the conjunction, respectively) leading to a leaf node labeled “0” and a branch labeled “1”/“0” leading to another non-leaf node, except for the lowest non-leaf node, that has a branch leading to a leaf node labeled “1”. c. The smallest possible decision tree consistent with a hypothesis in Pk contains 2 k+1^ − 1 nodes, since we must represent such a hypothesis with a complete binary tree with k + 1 levels. The lowest level contains 2k^ leaf nodes, and every non-leaf node in the next higher level has one child labeled “0” and one child labeled “1”. d. Since the size of the smallest possible tree in each case depends only on k and not n, we can reasonably conclude that a decision tree learning algorithm is a good choice in situations where we believe many of the features are irrelevant. Nevertheless, if we are stuck with Pk as our hypothesis space, decision tree learning is a bad choice. The fact that the smallest possible tree consistent with a hypothesis from Pk has size exponential in k implies that any decision tree learning algorithm must take a long time (computationally) to find a good hypothesis.
