Download Exam II Practice Problems - Machine Learning | COSC 6342 and more Exams Computer Science in PDF only on Docsity!
Exam
COSC 6342 Machine Learning
March 26, 2009
Your Name:
Your SSN:
Problem 1 [12]: Decision Trees (and kNN)
Problem 2 [5]: DENCLUE
Problem 3 [12]: Non-Parametric Density Estimation
Problem 4 [7]: Editing and Condesing
Problem 5 [8]: PCA
Problem 6 [10]: DBSCAN and Density-based Clustering
Problem 7 [9]: EM and Mixtures of Gaussians
Σ [63]:
Grade:
The exam is “open books and notes” and you have 75 minutes to complete
the exam. The exam is slightly too long. The exam will count approx. 24%
towards the course grade.
1) Decision Trees (and kNN) [12]
a) Compare decision trees with kNN to solve classification problems. What are the main differences between these two approaches? [5]
kNN Decision Tree
- Lazy learner - Learn model (tree)
- Local model - Global model
- Distance based - Based on attribute order
- Voronoi convex polygon decision boundary - Rectangle decision boundaries - Hierarchical learning strategy
b) We would like to predict the gender of a person based on two binary attributes: leg- cover (pants or skirts) and beard (beard or bare-faced). We assume we have a data set of 20000 individuals, 16000 of which are male and 4000 of which are female. 80% of the 16000 males are barefaced. Skirts are present on 50% of the females. All females are bare-faced and no male wears a skirt. i) Compute the information gain of using the attribute leg-cover for predicting gender! Just giving the formula that computes the information gain is fine; you do not need to compute the exact value of the formula! Use H as the entropy function in your formula (e.g. H(1/3,2/3) is the entropy that 1/3 of the examples belong to class1 and 2/3 of the examples belong to class 2). [2] ii) Computer the information gain of using the attribute beard to predict gender! [2]
i) Gain(D, leg-cover) = H(1/5, 4/5) – (1/10)H(1, 0) – (9/10)H(1/9, 8/9) ii) Gain(D, beard) = H(1/5, 4/5) – 4/25 H(0,1) – 21/25 H(16/21, 5/21)**
_() This question doesn’t require you to compute the exact value but you have to write the formulas in above forms to get credit._*
c) Why do decision tree learning algorithms grow the entire tree and then apply pruning techniques to trim the tree to obtain a tree of smaller size? [3]
- It grows the entire tree to fit training data (search as much detail as possible in hypothesis space).
- Then it prunes the tree to avoid over-fitting.
2) DENCLUE [5]
What role do non-parametric density functions play for the DENCLUE clustering algorithm? Give a description how the DENCLUE algorithm clusters a data set. Limit your answer to the second question to at most 6 sentences.
b) What is the main difference between the Gaussian Kernel Density function approach as described on page 157 of the textbook and the k-nearest Neighbor Density Estimator that has been described in Section 8.2.3. [3]
- Gaussian Kernel density approaches uses a fixed, global width while the kNN estimators employs a local, variable width that that corresponds to the k- nearest neighbor distance (if the density is measure in point v dk(v) determines the width used when measuring the density in v)
c) What advantages you see in using non-parametric density estimation approach compared to parametric density approaches, such as multivariate Gaussians? [3] Non-parametric density estimation approach:
- Doesn’t have to make assumption about model.
- Can estimate more complex models/shapes than parametric density approaches.
- Uses a non-global, Regional/case based approach to measure density
- Is easy to parallize on a single instruction multiple data (SMID), each processor stores 1 training function values for that instance.
4) Editing/Condensing/Toussaint Paper [7]
a) Both editing and condensing a are popular in conjunction with kNN classifiers. What is the goal of dataset editing? What is the goal of dataset condensing? [3]
- Editing: enhancing accuracy
- Condensing: improving speed by reducing the size of the dataset
b) Give a sketch of an algorithm that uses Voronoi diagrams (or their dual Delaunay graphs) for condensing a classification dataset![4]
- Voronoi condensing diagram retains the points whose neighbors are of opposite class and removes the points whose neighbors are of the same class. This is done by Delaunay triangulation.
5) PCA [8]
a) What is the goal of Principal Component Analysis (PCA)? Limit your answer to at most 5 sentences. [4]
- Its goal is computing most meaningful basis to represent data in a lower dimensional space.
b) The eigenvectors chosen to form the transformation wT^ that reduces dataset dimensionality in PCA have to orthonormal. What does this mean? Why is it desirable that the selected eigenvectors are orthonormal? [4]
- Othonomal means their covariance is equal to 0. Other answers may receive full credit.
- If they are dependent (not orthonormal), their contribution to the variance will be less; e.g. two variable with correlation 1 contribute as much to the density as just using one of the two variable.
6) DBSCAN & Density-based Clustering [10]
a) What the characteristics of objects that are classified as outliers by DBSCAN? [2]
- They are not in the radius of a core point.
- Or they are not density reachable from any core point.
b) How does DBSCAN form clusters? Limit your answer to at most 5 sentences [3]
- It specifies two parameters: radius (Eps) and MinPts (number of points).
- Then, it finds the core points , which have more than MinPts within Eps, the border points , which have fewer than MinPts within Eps but is in the neighborhood of a core point, and the noise points , which are not a core point or a border point.
- Any two points x and y to be density connected if there exists a core point z, such that both x and y are density reachable from z.
- Next, the method iterates from each core point, and finds all other points density connected to it; all such points belong to the same cluster.
c) DBSCAN does not well to cluster datasets that have clusters of varying densities. What is the explanation for that [2]?
- Because there is no global parameter setting for radius (Eps) or number of points (MinPts) that captures all the clusters. If we try to caputure high
