Image Processing: Lighting and Histogram Equalization - Prof. David Jacobs, Study notes of Computer Science

Download Image Processing: Lighting and Histogram Equalization - Prof. David Jacobs and more Study notes Computer Science in PDF only on Docsity!

Lighting affects appearance

Image Normalization

Global Histogram Equalization. Make two images have same histogram. Or, pick a standard histogram, and make adjust each image to have that histogram. Apply monotonic transform to intensities. Additive and multiplicative normalization. Subtract mean intensity, divide by total magnitude of result. Local Normalized cross-correlation: Normalize windows and thencompare with SSD. Normalize intensity and first derivatives -> direction of gradient. Normalize filter outputs: eg., Gabor Jets.

How do we represent light? (2)

Environment map: l (θ,φ)

• Light from all directions
• Diffuse or point sources
• Still distant
• Still no cast shadows.
• Example: outdoors (sky and sun)

Sky

`

Lambertian + Point Source

Surface normal

Light θθθθ

l ˆ ˆ issurface normal

is

isradiance

max( 0 , ( ˆ)

isintensityoflight

isdirectionoflight

n

albedo

i

i l n

l

l l l l

^

Lambertian, point sources, no shadows. (Shashua, Moses)

Whiteboard

Solution linear

Linear ambiguity in recovering scaled

normals

Lighting not known.

Recognition by linear combinations.

Basis

P=(x,y) means P = x(1,0)+y(0,1) Similarly:

• • • 

= +

cos( 2 ) sin( 2 )

( ) cos( ) sin( )

21 22

11 12 θ θ

θ θ θ

a a

f a a

Note, I’m showing non-standard basis, these are from basis using complex functions.

Example

cos(θ ) cosθ sin θ

, , such that :

1 2

1 2

c a a

c a a

• = +

∀ ∃

Orthonormal Basis

||(1,0)||=||(0,1)||= (1,0).(0,1)= Similarly we use normal basis elements eg:

While, eg:

= 

π θ θ θ θ

θ 2 0 cos( ) cos^2 cos( )

cos( ) d

 =

π θ θ θ

2 0 cos sin d 0

2D Example

Convolution

f ( x ) = g ∗ h =  g ( x − x 0 ) h ( x 0 ) dx 0

Imagine that we generate a point in f by

centering h over the corresponding point in

g , then multiplying g and h together, and

integrating.

Convolution Theorem

f g T F * G

− 1 ⊗ =

• F,G are transform of f,g

That is, F contains coefficients, when

we write f as linear combinations of

harmonic basis.

Examples

(cos. 2 cos 2. 1 cos 3 )?

cos?

cos cos 2?

cos cos?

• • ⊗ =

⊗ =

⊗ =

⊗ =

f

f

θ θ θ

θ

θ θ

θ θ

Low-pass filter removes low frequencies from

signal. Hi-pass filter removes high

frequencies. Examples?

Shadows

Attached Shadow

Cast Shadow

Images

Lighting Reflectance

(^00 1 2 )

1

(^00 1 2 )

1

2

r

Lighting to Reflectance: Intuition

++++^ ++++

Harmonic Transform of Kernel

k ( ) θ max(cos θ , 0) (^) n 0 k hn n 0

∞

= = (^) 

2 1 (^ 2)!^ (2^ 1)

1 3 ( 1) 2, even 2 ( 1)!( 1)! 0 2, od

2

2

d

2

0

n n n

n n (^) n n n

k

n

n

n

π

π

π

• (^) − + − ≥ −

     = (^)      

=

≥

=

0. 0.

0. 0.037 (^) 0.014 0. 0

0.

1

1.

0 1 2 3 4 5 6 7 8

Amplitudes of Kernel

n

A n

Energy of Lambertian Kernel in low order harmonics

Accumulated Energy

37.

87. 99.2 99.81 99.93 99.

0

20

40

60

80

100

120

0 1 2 4 6 8

Reflectance Functions Near

Low-dimensional Linear Subspace

0

( )

n nm nm nm n m n

r k l K L h = =−

= ∗ = (^)  

Yields 9D linear subspace.

  = =−

≈

2

0

( ) n

n

m n

Knm Lnm hnm

Forming Harmonic Images b (^) nm (p)= rnm (X,Y,Z)

λ λZ λX λY

2 (Z -X -Y )^2 2 2 (X -Y )^2 2 λXY λXZ λYZ

Compare this to 3D Subspace

λZ λX λY

Accuracy of Approximation of Images

Normals present to varying amounts.

Albedo makes some pixels more important.

Worst case approximation arbitrarily bad.

“Average” case approximation should be

good.

Models

Query

Find Pose

Compare

Vector: I

Matrix: B

Harmonic Images