Motion and Optical Flow, Lecture notes of Optics

Download Motion and Optical Flow and more Lecture notes Optics in PDF only on Docsity!

Motion and Optical Flow

Slides from Ce Liu, Steve Seitz, Larry Zitnick, Ali Farhadi

We live in a moving world

• Perceiving, understanding and predicting motion is an important part of our daily lives

Motion and perceptual organization

• Even “impoverished” motion data can evoke a

strong percept

G. Johansson, “Visual Perception of Biological Motion and a Model For Its Analysis", Perception and Psychophysics 14, 201-211, 1973.

Seeing motion from a static picture?

http://www.ritsumei.ac.jp/~akitaoka/index-e.html

How is this possible?

• The true mechanism is to

be revealed

• FMRI data suggest that

illusion is related to some

component of eye

movements

• We don’t expect computer

vision to “see” motion from

these stimuli, yet

What do you see?

The cause of motion

• Three factors in imaging process
  • Light
  • Object
  • Camera
• Varying either of them causes motion
  • Static camera, moving objects (surveillance)
  • Moving camera, static scene (3D capture)
  • Moving camera, moving scene (sports, movie)
  • Static camera, moving objects, moving light (time lapse)

Motion scenarios (priors)

Static camera, moving scene Moving camera, static scene Moving camera, moving scene Static camera, moving scene, moving light

How can we recover motion?

Recovering motion

• Feature-tracking
  • Extract visual features (corners, textured areas) and “track” them over multiple frames
• Optical flow
  • Recover image motion at each pixel from spatio-temporal image brightness variations (optical flow) B. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence , pp. 674–679, 1981. Two problems, one registration method

Feature tracking

• Given two subsequent frames, estimate the point

translation

• Key assumptions of Lucas-Kanade Tracker
  • Brightness constancy: projection of the same point looks the same in every frame
  • Small motion: points do not move very far
  • Spatial coherence: points move like their neighbors I ( x , y , t ) I ( x , y , t+1 )

x y t I ( x + u , y + v , t + 1 ) ≈ I ( x , y , t )+ I ⋅ u + I ⋅ v + I

• Brightness Constancy Equation: I ( x , y , t ) = I ( x + u , y + v , t + 1 ) Take Taylor expansion of I(x+u, y+v, t+1) at (x,y,t) to linearize the right side:

The brightness constancy constraint

I ( x , y , t ) I ( x , y , t+1 )

x y t

So: I u I v I

Image derivative along x I [u v] I 0 t T

x y t I ( x + u , y + v , t + 1 )− I ( x , y , t ) = + I ⋅ u + I ⋅ v + I Difference over frames

The aperture problem

Actual motion

The aperture problem

Perceived motion