Peekaboom: A Game for Object Localization and CAPTCHA Application, Study notes of Computers and Information technologies

Download Peekaboom: A Game for Object Localization and CAPTCHA Application and more Study notes Computers and Information technologies in PDF only on Docsity!

PeekaboomPeekaboom:

A Game for :

A Game for

Locating Objects in ImagesLocating Objects in ImagesB

L

Ah

R

L

d M

l Bl

By Luis von Ahn, Ruoran Liu, and Manuel Blum

ESP GameESP GameESP GameESP Game

y

Only attaches labels to random images

y

g

from the internet

y

Does not specify location of image

oes

ot spec y ocat o

o

age

◦

Thus, insufficient for training computer visionalgorithms

g

y

Given label can we identify where label

y

Given label, can we identify where labelactually is?

Sample Boom ImageSample Boom ImageSample Boom ImageSample Boom Image

Sample Peek ImageSample Peek ImageSample Peek ImageSample Peek Image

BUSH

Data that is collected for trainingData that is collected for trainingData that is collected for trainingData that is collected for training

y

Which pixels are necessary to guess the word?

y

From hints: what is the relation between wordand image?

y

Data from pings: which pixels are inside theobject?

y

What are the most relevant aspects of theobject?El

f

d

y

El

imination of poor image-word pairs

Dealing with single player gamesDealing with single player gamesDealing with single player gamesDealing with single player games

y

Bot can easily simulate a Boom from

y

previously saved human clicks

y

S

imulating Peek is much harder S

u at

g

ee

s

uc

a

e

Object Bounding BoxesObject Bounding BoxesObject Bounding BoxesObject Bounding Boxes

y

We can use Peekaboom results to establish tags for keywords in images

y

Example below: eyes and nose

ResultsResultsResultsResults

y

Is this an effective way to collect data?

y

Yes!

y

Game is enjoyable

y

Game is enjoyable^ ◦

Each person played average of 158.7 images ◦

That’s 72 96 minutes per person in one ◦

That s 72.96 minutes per person in onemonth!^ x

User review:

“This game is like crack. I’ve been

User review: This game is like crack. I ve beenPeekaboom-free for 32 hours…”

DiscussionDiscussionDiscussionDiscussion

y

What are some weaknesses of Peekaboom?

y

Ca

n you think of any other applications of

Ca

you t

o a y ot e

app cat o s o

Peekaboom?

CAPTCHA:CAPTCHA:

Using HardUsing Hard

AIAI

CAPTCHA:CAPTCHA:

Using HardUsing Hard

AIAI

Problems for SecurityProblems for SecurityBy Luis von Ahn Manuel Blum Nicholas J HopperBy Luis von Ahn, Manuel Blum, Nicholas J. Hopper,John Langford

More ExamplesMore ExamplesMore ExamplesMore Examples

More ExamplesMore ExamplesMore ExamplesMore Examples

What is a CAPTCHA?What is a CAPTCHA?What is a CAPTCHA?What is a CAPTCHA?

y

What kind of AI problems do we want to

p

use?

Hard problems

.^

a

p ob e

s

◦

Hard w.r.t. complexity

◦

AI community agrees it

’s hard

AI community agrees it s hard

Useful problems

What is a CAPTCHA?What is a CAPTCHA?What is a CAPTCHA?What is a CAPTCHA?

y

The fact that CAPTCHA is based on a hard AI problem gives us a win-winsituation^ ◦

Either someone figures out a bot to solveCAPTCHAs (solving a hard, useful AI problem) ◦

Or there is a way to tell humans from computers, which is useful for security