Model Fitting: Finding the Best Fit for Data using Linear Regression and Other Techniques , Study notes of Computer Science

Download Model Fitting: Finding the Best Fit for Data using Linear Regression and Other Techniques and more Study notes Computer Science in PDF only on Docsity!

Model Fitting

Fitting data to a model•^

Practical computer vision involves lots of fitting of data tomodels

-^

Tasks– Fit image features to projection of hypothetical shape

• Fit lines, ellipses, etc.
  • Determine the camera projection matrix– Determine motion of objects in video– Determine 3-D coordinates of world points– … many others -^

All these involve fitting data obtained from images tomodels

-^

Simplest example of model fitting problem– Linear regression

•^

More examples on the board– higher dimensions– Matrix entries …

-^

In all these we need to find the models and theirparameters that best explain the data

-^

General part of “vision as inference”

-^

Approaches– Based on fitting: today– Based on sampling model-space: e.g., Hough transform– Combination

Relationships among Variables:Interpretations

One variable is used to “explain” another variable

X

Variable

Independent VariableExplaining VariableExogenous Variable

Predictor Variable

Y

Variable

Dependent VariableResponse VariableEndogenous VariableCriterion Variable

1

slope:

b

intercept

a

bX a

Y

=

:s

prediction

errorless had we If

X

Simple Least-Squares RegressionY

X

Y

We will end up being reasonably confidentthat the true regression line is somewherein the indicated region.

X

Y

Estimated Regression Line

X

Y

Estimated Regression Line

i

i

i^

y

y

e^

ˆ −

=

y^ i ˆyi

x^ i

bX a Y^

= ˆ

:

Line

Regression the of

Equation

How do we find a and b?

(^

)^

[^

]

(^

)

∑

∑

=

=

N^1 i

2

N^1 i

2

iduals

errors/res

squared of

sum the

minimize to ,

Find

a

bx

y

e

b a

i

i

In Least-Squares Regression:i

In Least-Squares Regression:

(^

(^

)^

X b Y a X X

Y

Y

X

X

b

N i

i

N i

i

i

= =^

1

2

1

=^

=

=^

=

=

⎞ ⎟ ⎠

⎛ ⎜ ⎝ −

⎞ ⎟ ⎠

⎛⎞ ⎜⎟ ⎝⎠

⎛ ⎜ ⎝ −

=

N i

N i

i

i

N i

N i

i

N i

i

i i

X

X

N

Y X Y X N b

1

2

1

2

1

1

1

ComputationalFormula

Linear models•^

All models considered are “linear”

-^

(does not mean straight lines)

-^

Means that we can separate the “structure” and“parameters” as a matrix-vector product

-^

“structure” forms a matrix and “parameters” a vector

-^

Goal of model fitting: find the parameters

-^

Question: Is the Gaussian model a linear one?

Linear Systems

A

x

b

Square system:

• unique solution• Gaussian elimination

Rectangular system ??

• underconstrained:

infinity of solutions Minimize |Ax-b|

2

• overconstrained:

no solution

A

x

b

Least Squares for more complex models•^

Number of equations and unknowns may not match

-^

Data may have noise

-^

Look for solution by minimizing some cost function

-^

Simplest and most intuitive cost function: ||

Ax - b

•^

Define for each data point

x

a residuali

r

i

•^

Minimize

ri

ri with respect toi^

x

l

•^

r^ i

r

=i

(Aj

xij

-bj

).i

k^

(A

ik

xk

-b

)i

SVD and Pseudo-Inverse