Download Cross Correlation - Lecture Slides | CS 510 and more Study notes Computer Science in PDF only on Docsity!
Cross Correlation
Lecture #16March 2, 2009
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Cross-Correlation and Rotation^ z
The brute force approach to making correlation insensitiveto rotation is to generate R templates at R different angles.–
Increase complexity by the factor R
-^
Only accurate if R is fairly large
z
If you could guess the orientation, you could apply onlyone template per pixel location–
So how do you measure the orientation of a pixel?
-^
What does it even mean?
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Image as Surface z
View the image as a 3D surface–
For every (x,y) pixel location, the intensity can be thought of asthe z (height) value.
z
Color images are 5D surfaces - too hard to think about. z
Color can also be thought of as 3 3D surfaces
-^
Pretend the surface is continuous
z
Every point on the image surface has a direction ofmaximum change (remember your multivariate calculus?),and a magnitude of change in that direction
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009
Computer Graphics, Spring 2009
Image Edges z
To compute the direction and magnitude of change,compute the magnitude of change in any two orthogonaldirections and interpolate–
Again, this assumes a continuous surface
z
Compute the magnitude of change in the X & Y directions:–
dx(x,y) = I(x,y) - I(x-1,y)
-^
dy(x,y) = I(x,y) - I(x,y-1)
z
The magnitude of change in the maximal direction is then
z
The orientation of maximal change is
2
2
dy
dx
mag
−
2
2
1
cos
dy
dx
dy
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Rotation-Free Cross Correlation^ z
Make sure template is centered on an edge–
Measure it’s orientation
z
For every pixel location:–
compute edge orientation at pixel
-^
rotate template until pixel edge matches the orientation of thetemplate edge
z
rotate using bilinear interpolation
-^
correlate template with image
z
Makes correlation insensitive to rotation–
If edge direction estimates are accurate
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Estimating Edge Orientation z
Problem: images are not really continuoussurfaces–
estimates of dx, dy based on grid sampling
note that if accurate,
estimating derivatives from two values is highly errorprone.
(^
)^
(
)^
(^
)^
(
)
y
x
I
y
x
I y x I y x
I^
,^
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
- Computer Graphics, Spring 2009
Computer Graphics, Spring 2009
z
We want to compute a real-valued function
-^
The partial derivatives dx & dy
z
All we have are samples at equidistant points
z
So model the function in terms of its Taylor seriesexpansion:
Accurate Edge Estimation
(^
)^
L
x f h x f h x f h x
f^
2
1
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Accurate (II)^ z
So look at the equations for f(x+h) and f(x-h):
z
Subtract (2) from (1) to get
z
And solve for f’(x)
(^
)^
L
x
f
h
x f h
x f
h
x f^
(^
)
( )
( )
( )
L
x
f
h
x f h
x f
h
x f^
(^
)^
(^
)^
( )
L
x f h h x f h x
f^
( )
(^
)^
(^
)^
L
′^
h
h
x f
h
x f
x f^
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Accurate (III) z
So the best ±1 mask is [-1,0,1], from
z
As an exercise, the best ±2 mask is [1,-8,0,8,-1]
(^
)^
(^
)^
(^
)
,^
y
x f y x f y x
f
d dx
(^
)^
(^
)^
(^
)
(^
)^
(^
)
,^
y x f y x f y x
f y x f y x f
d dx
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
- Computer Graphics, Spring 2009
Computer Graphics, Spring 2009
Stable Edges (III) z
Of course, pixels are still noisy and pixels arerelated to the rows below and above, and…
z
We don’t care if all edges are multiplied by aconstant.
z
The Sobel Edge Masks
Dx
⎡^ ⎢⎢⎢⎣
Dy
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Discussion
z
Correlation is the base technique against which otherimage matching algorithms are compared
z
It is sensitive to translation, rotation, and scale
-^
cross-correlation compensates for translation
-^
rotation-free cross-correlation compensates for rotation too
-^
both are expensive
z
One approach to the homework:
-^
Implement simple cross-correlation
-^
Add rotation
-^
Build machinery for appling image transformations to templates
-^
Where score is good, search for scale changes that improve it.
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Matching^ z
The Fourier Transform converts spatial images into frequencies
and
phases
. Can we use these to match
images?
z
The Fourier transform is deterministic, so if two imagesare identical, so are their frequencies and phases.
z
Can we tell if two images are similar?
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
- Computer Graphics, Spring 2009
Computer Graphics, Spring 2009
Example
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Mosaicing (II)
z
How do we Mosaic images?–
We could do cross-correlation, but both images arelarge, so this is expensive.
-^
Alternatively, we could take the Fourier transform ofboth (and the complex conjugate of one) and computethe phase shift using:
(^
)^
(^
(^
)^
(^
(^
) 0
0 ,
2
1
1
,^
vy ux i e
v u
F v u F
v u
F v u F
π
=
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Mosaicing (III)
z
What will be the result? If the images are identical (otherthan the shift) and you sum the power at each phase acrossfrequencies u,v, you will find that all the energy lies atsingle phase corresponding to the shift x
,y 0
0
z
What if they are similar but not identical? Then most of thepower lies at a single phase, so perform the samecalculation and then do peak detection.
z
Is this cheaper than cross-correlation?
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
- Computer Graphics, Spring 2009
Computer Graphics, Spring 2009
Fourier Methods^ z
Mosaicing involves matching a whole to a shiftedwhole–
What if we want to find a small template in a largerimage?
z
Will Fourier matching work? z
Why or why not?
For what other types of problems does Fouriermatching work?
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Back to Correlation^ z
I said that if two images are similar, then phase correlationsays that the maximum phase value corresponds to the bestshift x
, y 0
z
What is the relationship between the height of the peak andthe correlation score of the images under that shift?
z
Parseval’s theorem says that:
z
So normalizing source images normalizes frequency space,and the height of the peak is the correlation score
(^
∞ ∞−
∞ ∞−
=^
df f H
dx x h
2
2
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Similarity Transforms
z
So we can compensate for image translation using theshift theorem
z
Can we compensate for
-^
image rotation?
Yes
-^
image scaling?
Yes
-^
perspective distortion?
No
z
How? By using the shift theorem again...
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
- Computer Graphics, Spring 2009
Computer Graphics, Spring 2009
Rotation
Translation
z
Our goal is to apply a coordinate transformation to theoriginal spatial domain image such that a change inrotation becomes a change in translation.
z
How? By converting from Cartesian Coordinates to PolarCoordinates.–
Remember that the Polar Coordinates of a point in 2D are itsdistance from the origin
d
and the angle of the vector from the
origin to the point
φ
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Polar Coordinates
Image (3x3)
Cartesian[0,0]=255[0,1]=250[0,2]=224[1,0]=225[1,1]=0[1,2]=255[2,0]=249[2,1]=255[2,2]=
Polar[0,0]=
[1,0]=
[0,45]=
[1,45]=
[0,90]=
[1,90]=
[0,135]=0 [1,135]=255[0,180]=0 [1,180]=225[0,225]=0 [1,225]=249[0,270]=0 [1,270]=255[0,315]=0 [1,315]=
Polar coordinates imply an
image resampling - use bilinear interpolation
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510 -CS
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Polar Coordinates (cont.)
z
Why convert into polar coordinates?–
Because a cartesian rotation becomes a polartranslation
z
2nd coordinate -- angle -- advances…
Apply Fourier transform to polar image, findmaximum shift
z
Shift in distance coordinate must be zero!
Shift implies rotation in cartesian space.
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
- Computer Graphics, Spring 2009
Computer Graphics, Spring 2009
Scale
Shift
z
What if one image is scaled relative to another?
z
Convert image to LogPolar Coordinates
-^
Just like Polar Coordinates, except that the distance coordinate isexpressed by its logarithm
z
Now a shift in the distance coordinate is equivalent to ascale change
z
Note that the shift theorem finds shifts in both dimensions,so you can match images that are both rotated and scaled.
©^
Bruce A. Draper & J. Ross Beveridge, March 2 2009
CS510CS510 -
- Computer Graphics, Spring 2009Computer Graphics, Spring 2009
Limitations
z
We can only match rotated and/or scaled image
if we know
the point they are rotated and/or scaled about!
z
We can match translated images or rotated & scaled images,but not both.
z
There is an informal, iterative technique for handling smallchanges in translation, rotation & scale
-^
Assume no translation, find rotation & scale shift, apply it
-^
Assume no rotation/scale, find translation, apply it
-^
Repeat until peak in shift theorem reaches a threshold, or fails toimprove
