Download Machine Learning Exam and more Exams Machine Learning in PDF only on Docsity!
Machine Learning Exam
12/01/2018, 15-18h
Please answer each question in a separate sheet of paper and present all responses in a clear and ordered manner, with a brief justification of each step.
- (3 points) x 1 x 2 y 1 1 0 1 -1 - 1 2 1 Consider the table above. We wish to predict the variable y knowing the features x 1 , x 2 , and using a linear regression model yˆ = β 1 x 1 + β 2 x 2.
(a) write the normal equations for the sum of squared errors (SSE) criterion, and obtain the optimal coefficients βˆi, i = 1, 2.
(b) evaluate the minimum value of the SSE criterion.
- (3 points) Consider the following binary classification problem. Let the observation x > 0 be a realization of a random variable generated by one of the following density functions
class ω 0 : p(x|ω 0 ) = α 0 e−α^0 x^ class ω 1 : p(x|ω 1 ) = α 1 e−α^1 x,
where α 1 , α 0 , (α 1 > α 0 ), are known parameters.
(a) Define the Bayes classifier for this problem, assuming that both classes ω 0 , ω 1 are equally probable and that all the decision errors have the same penalty.
(b) Find the decision regions of the Bayes classifier.
- (4 points) Consider the following multilayer perceptron where all the units have as activation function g(s) = (^) 1+^1 e−s with derivative g′(s) = g(s)(1 − g(s)). Also consider the following training set.
x 1 x 2 y 1 y 2 − 1 1 0. 5 0. 5 1 − 1 0 − 1
(a) Draw the backpropagation network. Don’t forget to include the gains of all the branches, as well as the input and output variables.
(b) Compute the value of the weight b and d after the first update using backpropagation in real-time mode. The cost function is the sum of squared errors and the training is performed using fixed step size with parameter η = 0.2. Assume that, initially, all the weights are equal to 0.1.
- (4 points) Consider a binary classification problem with non-separable data in R^2. The data becomes separable if we apply the following nonlinear mapping from input space to feature space, given by
(x 1 , x 2 ) → (x 1 x 2 , x 1 (x 1 − x 2 )).
Assume you know that points (0, 0) from class B and (1, 2) from class A are support vectors in input space and that (3, −2) is a support vector from class A in feature space.
(a) Find the equations for the SVM classification boundary and margins in feature space and plot them together with the support vectors.
(b) Compute the margin in feature space.
(c) Find and plot the margins and the boundaries in input space.
(d) Plot the decision regions for classes A and B in input space.
(e) Explain the differences between the SVM for separable (hard margin) and non separable (soft margin) data, addressing the cost function and location of the support vectors with respect to the margin boundaries.
- (3 points)
x 1 x 2 x 3 y count 0 0 0 A 1 0 0 1 A 1 0 1 0 B 1 0 1 1 A 1 1 0 0 B 1 1 0 1 B 3 1 1 0 B 0 1 1 1 A 3 Consider the table above with binary attributes, labels and number of occurrences of each example. Grow a decision tree for this data set using the ID3 algorithm.
