Brightness Constancy and Optical Flow, Study notes of Optics

Download Brightness Constancy and Optical Flow and more Study notes Optics in PDF only on Docsity!

Brightness Constancy

16-385 Computer Vision (Kris Kitani) Carnegie Mellon University

Optical Flow

Problem Definition Assumptions Brightness constancy Small motion Given two consecutive image frames, estimate the motion of each pixel

Brightness constancy

I(x, y, 1) I(x, y, 2)

I(x, y, k) (x(1), y(1)) (x(2), y(2)) (x(k), y(k)) Scene point moving through image sequence Assumption 1

Brightness constancy

I(x, y, 1) I(x, y, 2)

I(x, y, k) (x(1), y(1)) (x(2), y(2)) Assumption:Brightness of the point will remain the same (x(k), y(k)) Scene point moving through image sequence Assumption 1

Small motion

I(x, y, t) I(x, y, t + t) Assumption 2

Small motion

I(x, y, t) (x, y) (x + ut, y + vt) (x, y) I(x, y, t + t) Assumption 2

Small motion

I(x, y, t) (x, y) (x + ut, y + vt) (x, y) Optical flow (velocities): (^) (u, v) Displacement: (x, y) = (ut, vt)

I(x + ut, y + vt, t + t) = I(x, y, t)

… the brightness between two consecutive image frames is the same I(x, y, t + t) For a really small space-time step … Assumption 2

dI dt = @I @x dx dt

@I @y dy dt

@I @t = 0 total derivative partial derivative Equality is not obvious. Where does this come from? These assumptions yield the …

Brightness Constancy Equation

I(x + ut, y + vt, t + t) = I(x, y, t) For small space-time step, brightness of a point is the same

I(x + ut, y + vt, t + t) = I(x, y, t) For small space-time step, brightness of a point is the same Insight: If the time step is really small, we can linearize the intensity function

I(x + ut, y + vt, t + t) = I(x, y, t) Multivariable Taylor Series Expansion (First order approximation, two variables) f (x, y) ⇡ f (a, b) + f x (a, b)(x a) f y (a, b)(y b) assuming small motion I(x, y, t) + @I @x x + @I @y y + @I @t t = I(x, y, t)

I(x + ut, y + vt, t + t) = I(x, y, t) assuming small motion I(x, y, t) + @I @x x + @I @y y + @I @t t = I(x, y, t) cancel terms Multivariable Taylor Series Expansion (First order approximation, two variables) f (x, y) ⇡ f (a, b) + f x (a, b)(x a) f y (a, b)(y b) fixed point partial derivative

I(x + ut, y + vt, t + t) = I(x, y, t) assuming small motion divide by take limit t t! 0 I(x, y, t) + @I @x x + @I @y y + @I @t t = I(x, y, t) @I @x x + @I @y y + @I @t t = 0 Multivariable Taylor Series Expansion (First order approximation, two variables) f (x, y) ⇡ f (a, b) + f x (a, b)(x a) f y (a, b)(y b)

I(x + ut, y + vt, t + t) = I(x, y, t) assuming small motion @I @x dx dt

@I @y dy dt

@I @t = 0 divide by take limit t t! 0 I(x, y, t) + @I @x x + @I @y y + @I @t t = I(x, y, t) @I @x x + @I @y y + @I @t t = 0 Multivariable Taylor Series Expansion (First order approximation, two variables) f (x, y) ⇡ f (a, b) + f x (a, b)(x a) f y (a, b)(y b)