1 | P a g e
CS434 – Final Exam Prep: Machine
Learning and Data Mining.
1. Softmax function - Answers ✅✅ Takes in a d-dimensional vector of exponentiated activations
which contains the activation for each of d classes, with each class activation divided by the
sum of all d activations. This outputs a normalized d-dimensional vector such that all elements
sum to one.
2. In multiclass logistic regression, the total number of learned weight vectors for a C class
problem. - Answers ✅✅ C or C-1
3. Artificial neurons - Answers ✅✅ Behave very differently than their biological counterpart
4. Capability of single-activation neuron - Answers ✅✅ Can represent non-linear boundaries in
classification problems.
5. ReLUs - Answers ✅✅ Tends to converge fast because they do not saturate for positive inputs
like Sigmoid.
6. Loss function - Answers ✅✅ Measures how big an error predicted by a network is from the
ground truth. Generally decreases when plotted versus number of epochs when training a
neural network.
7. Jacobian - Answers ✅✅ A matrix filled with partial derivatives of each dimension of vector v
with respect to each dimension of vector u. Thus, it's a high-dimensional gradient.
8. Backpropagation - Answers ✅✅ An efficient way to compute gradients of the loss with
respect to model parameters. Firstly, it is a reverse-mode differentiation and computes the
product of intermediate Jacobians from the output of the network backwards -- reducing cost
of matrix multiplication in computing gradients. Secondly and more important, the backwards
pass allows us to store and reuse loss gradients as we work our way backwards through the
network.
9. Neural networks - Answers ✅✅ Universal approximators of finite size
