Prepare for your exams
Get points
Guidelines and tips
Sell on Docsity
Docsity AI

Prepare for your exams

Study with the several resources on Docsity

Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips

Sell on Docsity

Docsity AI

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents

20 Points

For each uploaded document

Answer questions

5 Points

For each given answer (max 1 per day)

All the ways to get free points

Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Fall 2008 Lecture 4: Machine Learning - Logistic Regression & Perceptron Algorithm by S. A, Assignments of Computer Science

University of Southern California (USC)Computer Science

A part of the fall 2008 lecture notes for the machine learning (cs 567) course taught by sofus a. Macskassy at the university of southern california. The notes cover topics such as logistic regression, conditional probability distribution, and the perceptron algorithm. Students are introduced to the concept of learning conditional distributions and the use of logistic functions to estimate probabilities. The lecture also discusses the optimization of logistic regression using gradient descent and the difference between online and batch learning.

Typology: Assignments

Pre 2010

Uploaded on 02/24/2010

koofers-user-74z 🇺🇸

10 documents

1 / 44

This page cannot be seen from the preview

Don't miss anything!

Fall 2008 Lecture 4 - Sofus A. Macskassy1

Machine Learning (CS 567) Lecture 4

Fall 2008

Time: T-Th 5:00pm - 6:20pm

Location: GFS 118

Instructor: Sofus A. Macskassy ([email protected])

Office: SAL 216

Office hours: by appointment

Teaching assistant: Cheol Han ([email protected])

Office: SAL 229

Office hours: M 2-3, W 11-12

Class web page:

http://www-scf.usc.edu/~csci567/index.html

Discover Assignments of Computer Science University of Southern California (USC)

Partial preview of the text

Download Fall 2008 Lecture 4: Machine Learning - Logistic Regression & Perceptron Algorithm by S. A and more Assignments Computer Science in PDF only on Docsity!

Fall 2008 1 Lecture 4 - Sofus A. Macskassy

Machine Learning (CS 567) Lecture 4

Fall 2008

Time: T-Th 5:00pm - 6:20pm

Location: GFS 118

Instructor : Sofus A. Macskassy ([email protected])

Office: SAL 216

Office hours: by appointment

Teaching assistant : Cheol Han ([email protected])

Office: SAL 229

Office hours: M 2-3, W 11-

Class web page:

http://www-scf.usc.edu/~csci567/index.html

Fall 2008 2 Lecture 4 - Sofus A. Macskassy

Administrative: Home work

• Due at start of class on due date.

• 25% off if after class but before 8am next morning

• 50% off if received next day after 8am

• 100% off if not received next day

• If you have a valid excuse, let me know ASAP with proper

documentation. No exceptions.

• Grading compliant policy: if students have problems at

assignment grading, feel free to talk to the instructor/TA for

it. However, even if students only request re-grading one of

the answers, all this assignment will be checked and graded

again. (take the risk of possible losing total points.)

Fall 2008 4 Lecture 4 - Sofus A. Macskassy

Project Idea: Applied Learning

Take an interesting dataset Compare several learning approaches for prediction

decision trees
ANNs
instance-based methods
SVMs
boosting

Fall 2008 5 Lecture 4 - Sofus A. Macskassy

Project Idea: Improvements to

Learning Methods

There are many suggestions on how to improve various learning methods, both in books and in papers. Identify some suggestions and test them empirically.

Fall 2008 7 Lecture 4 - Sofus A. Macskassy

Text Classification

Easy to get lots of text: web, TREC data,

email (e.g., enron)

Predict topic, authorship, sentiment, style,

affect, attitude.

Fall 2008 8 Lecture 4 - Sofus A. Macskassy

Reinforcement Learning

Can be challenging to do well: generate data or

direct control

• Optimize gas well production

• Object tracking

• “Tag” grid world

• Rover control

• animal behavior experiments

Compare approaches (direct policy search, value

function learning, model-based, …)

Fall 2008 10 Lecture 4 - Sofus A. Macskassy

Project Methodology

Lots of good ideas for algorithms and domains.

The hard question is: “How will you evaluate it?”

Ultimately, you need to present more than one

algorithm (and perhaps more than one problem)

and you’ll need some way of saying what worked

better.

What’s the gold standard?

Fall 2008 11 CS 567 Lecture 3 - Sofus A. Macskassy

Lecture 4 Outline

• Linear Threshold Units

– Perceptron

– Logistic Regression

Fall 2008 13 CS 567 Lecture 3 - Sofus A. Macskassy

Alternatively…

w

w w

g g g

T

T T

w x

w w x

w x w x

x x x

otherwise

if 0

choose

C

C g x

Fall 2008 14 CS 567 Lecture 3 - Sofus A. Macskassy

Geometry

Fall 2008 16 CS 567 Lecture 3 - Sofus A. Macskassy

Pairwise Separation

ij

T

ij ij ij ij

g x w ,w  w x  w

don' t care otherwise

0 if

j i ij

C

g x

x

choose if

  x 

ij i

j i, g

C

Fall 2008 17 CS 567 Lecture 3 - Sofus A. Macskassy

• When learning a classifier, the natural way to

formulate the learning problem is the following:

– Given:

A set of N training examples {( x 1 ,y 1 ), ( x 2 ,y 2 ), …, ( x N,yN)}
A loss function L

– Find:

The weight vector w that minimizes the expected loss on the

training data

• In general, machine learning algorithms apply some

optimization algorithm to find a good hypothesis. In

this case, J is piecewise constant, which makes this

a difficult problem

J(w) =

N

X^ N i=

L(sgn(w ¢ xi); yi):

Machine Learning and Optimization

Fall 2008 19 Lecture 4 - Sofus A. Macskassy

Optimizing… what?

Hinge loss 0-1 loss

Fall 2008 20 CS 567 Lecture 3 - Sofus A. Macskassy

Optimizing… what?

w 0

Fall 2008 Lecture 4: Machine Learning - Logistic Regression & Perceptron Algorithm by S. A, Assignments of Computer Science

Related documents

Partial preview of the text

Download Fall 2008 Lecture 4: Machine Learning - Logistic Regression & Perceptron Algorithm by S. A and more Assignments Computer Science in PDF only on Docsity!

Machine Learning (CS 567) Lecture 4

Fall 2008

Time: T-Th 5:00pm - 6:20pm

Location: GFS 118

Instructor : Sofus A. Macskassy ([email protected])

Office: SAL 216

Office hours: by appointment

Teaching assistant : Cheol Han ([email protected])

Office: SAL 229

Office hours: M 2-3, W 11-

Class web page:

http://www-scf.usc.edu/~csci567/index.html

Administrative: Home work

• Due at start of class on due date.

• 25% off if after class but before 8am next morning

• 50% off if received next day after 8am

• 100% off if not received next day

• If you have a valid excuse, let me know ASAP with proper

documentation. No exceptions.

• Grading compliant policy: if students have problems at

assignment grading, feel free to talk to the instructor/TA for

it. However, even if students only request re-grading one of

the answers, all this assignment will be checked and graded

again. (take the risk of possible losing total points.)

Project Idea: Applied Learning

Project Idea: Improvements to

Learning Methods

Text Classification

Easy to get lots of text: web, TREC data,

email (e.g., enron)

Predict topic, authorship, sentiment, style,

affect, attitude.

Reinforcement Learning

Can be challenging to do well: generate data or

direct control

• Optimize gas well production

• Object tracking

• “Tag” grid world

• Rover control

• animal behavior experiments

Compare approaches (direct policy search, value

function learning, model-based, …)

Project Methodology

Lots of good ideas for algorithms and domains.

The hard question is: “How will you evaluate it?”

Ultimately, you need to present more than one

algorithm (and perhaps more than one problem)

and you’ll need some way of saying what worked

better.

What’s the gold standard?

Lecture 4 Outline

• Linear Threshold Units

– Perceptron

– Logistic Regression

Alternatively…

w

w w

w w

g g g

T

T

T T

w x

w w x

w x w x

x x x

otherwise

if 0

choose

C

C g x

Geometry

Pairwise Separation

ij

T

ij ij ij ij