Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Machine Learning Homework 1: Linear Models and SVM, Assignments of Computer Science
Information about the first homework assignment for the machine learning course (cs 5350/cs 6350). The assignment covers linear models, support vector machines (svm), and optimization problems. Students are required to complete exercises from prml, write answers to specific questions, and implement the perceptron algorithm with averaging extension or gradient descent for logistic regression. The document also provides instructions for handing in the assignments.
Typology: Assignments
1 / 1
Related documents
Partial preview of the text
Download Machine Learning Homework 1: Linear Models and SVM and more Assignments Computer Science in PDF only on Docsity!
Machine Learning (CS 5350/CS 6350) Due 01 Feb 2006
HW1: Linear models
Note: Please submit the written aspects of assignments in postscript or PDF format only. I highly recom- mend you use LATEX to prepare the assignments. A solution will be posted here after the due date. See http://www.cs.utah.edu/classes/cs5350/handin.html for handin instructions.
1 PRML Exercises
Complete the following exercises from PRML: 1.2, 1.36, 3.3, 4.1, 4.7, 7.2.
CS 6350 students also do 4.14.
2 Written Exercises
- Express the SVM classification problem as an unconstrained optimization problem. Compute the gradient (you may ignore the discontinuities). Compare and contrast this solution with the gradient of logistic regression and with the perceptron optimization problem.
- What distinguishes generative, conditional and discriminative methods (100-200 words)?
- Intuitively, why are large margins good?
- What is the VC dimension of the set of all degree three polynomials? Prove it.
3 Programming
Implement the perceptron algorithm with the averaging extension.
- Download training, development and test data from the web page (each row is an example, the first column is the class and the remaining columns are feature values).
- Run your perceptron using the raw data, data that has been normalized to have unit L1 norm, and data that has been normalized to have unit L2 norm. Which seems to work best?
- Plot error rates for the training, development and test data as a function of iteration (both the raw predictions and the averaged predictions).
- How well does the development performance track the test performance?
You should hand in: (A) your code; (B) results on raw, L1 and L2 normalized data; (C) error plots; (D) answers to questions.
CS 6350 Students: Repeat the previous exercise but using gradient descent for (regularized) logistic regression rather than the perceptron. Hint: You shouldn’t have to change very much code. Turn in everything that you did for the perceptron. Also, compare the performance of the two algorithms on the test data, and vary the regularization parameter for the logistic regression. Which seems better? Why?