CMSC 726 Homework 4
Due Date: Thursday, April 6, at the start of class
In this assignment, you will complete the exploration of classifier performance on your
dataset.
1. [40 points] Using WEKA, compare the performance of the new methods we have
seen (DecisionTrees, kNN, Neural Networks, SVMs) on your dataset.
(a) Turn in a learning curve, showing the performance of these algorithms, along with
your earlier results using logistic regression and naive Bayes from homework 2.
As before use 5-fold cross validation.
(b) For the largest size training set, turn in a table which has the average accuracy
and the standard deviation for each of the classifiers.
(c) Which performs best? Is this result statistically significant?
(d) Include a discussion of your results.
2. [60 points] In this question, you will implement a linear threshold unit (LTU) and
an ensemble method, and compare with your earlier performance using LR and NB.
(a) Implement your own version of a linear threshold unit (single node neural net-
work). You can implement any reasonable variation that you like. You may use
whatever language with which you are most comfortable (C, Java, Matlab, Perl,
etc.).
(b) Compare the performance with your LR and NB classifiers.
(c) Implement an ensemble method of your choice, using your LTU, LR and/or NB
classifiers as you like.
(d) Turn a discussion of your results and a printout of the main portions of your
implementation.