1
CS6780
Advanced Machine Learning
Spring 2019
Thorsten Joachims
Cornell University
Department of Computer Science
Outline of Today
•Who we are?
–Prof: Thorsten Joachims
–TAs: Aman Agarwal, Ashudeep Singh
•What is learning?
–Examples of machine learning (ML).
–What drives research in and use of ML today?
•Syllabus
–Topics and Methods
–Themes
•Administrivia
(One) Definition of Learning
•Definition [Mitchell]:
A computer program is said to learn from
•experience E with respect to some class of
•tasks T and
•performance measure P,
if its performance at tasks in T, as measured by P,
improves with experience E.
What is the goal of CS6780?
•PhD-level introduction to machine learning
–First or second ML class
•Broad, but deep along several key themes
•Enable your research in or with machine
learning
•Practice “soft” skills you need as researcher
Syllabus
•Supervised Batch Learning: model, decision theoretic foundation, model
selection, model assessment, empirical risk minimization
•Decision Trees: TDIDT, attribute selection, pruning and overfitting
•Statistical Learning Theory: generalization error bounds, VC dimension
•Large-Margin Methods: linear Rules, margin, Perceptron, SVMs
•Kernels: duality, non-linear rules, non-vectorialdata
•Deep Networks: multi-layer perceptrons, convolutions, pooling
•Structured Output Prediction: hidden Markov model, Viterbi, structural
SVMs, conditional random fields
•Probabilistic Models: generative vs. discriminative, maximum likelihood,
Bayesian inference
•Latent Variable Models: k-means clustering, mixture of Gaussians,
expectation-maximization algorithm, matrix factorization, embeddings
•Online Learning: experts, bandits, online convex optimization
•Causal Inference: interventional vs. observational data, treatment effects,
policy learning
Theme: Prediction and Action
•Building intelligent systems vs. analyzing
existing systems
–Prediction
–Intelligent action
–Guarantees on prediction/action quality
–Causality
