Linear Regression Analysis - Political Science - Lecture Slides, Slides of Political Science

Download Linear Regression Analysis - Political Science - Lecture Slides and more Slides Political Science in PDF only on Docsity!

Linear Regression Analysis

Regression Analysis

• Regression analysis is

the predominant

statistical tool used in

the social sciences

• Simple and versatile
• AKA: linear regression,

ordinary least squares,

OLS

• Central concept is fitting

a line through data to

describe relationships

between X and Y

Translating Math into English

• Linear model implies that the dependent variable is directly proportional to the independent variable.
• A theory implying that Y increases in direct proportion to an increase in X, implies a specific mathematical model of behavior - the linear model. - Example: Economic performance and incumbent vote

share

• ALL statements of relationships between variables imply a mathematical structure. - Even if we don’t like to phrase our theories in these terms,

they DO imply mathematical relationships

• Courses in regression analysis are about making this basic

linear model fit more nuanced theories

The Regression Parameters

• a = the intercept
  • the point where the line crosses the Y-axis.
  • (the value of the dependent variable when all of the

independent variables = 0)

• b = the slope
  • the increase in the dependent variable per unit change in the

independent variable (also known as the 'rise over the run')

• Ordinary Least Squares (OLS) is a method of finding the parameters a & b that define the line of best fit between variables - Line that provides the best explanation/prediction of the data - Determined by minimizing the squared errors around the line

Yi = a + bX (^) i + ei

Determining the Line of Best Fit

Finding the Line of Best Fit

TSS Y Y

ESS Y Y

USS Y Y

i

i

i i

= = −

= = −

= = −

∑

∑

∑

Total Sum of Squares

Explained Sum of Squares

Unexplained Sum of Squares

( )

( ^ )

( ^ )

2

2

2

Total Variation = Explained Variation + Unexplained Variation

The OLS Estimators for the Slope and Intercept

∑

∑

−

− − = (^2) ( )

( )( ) ˆ X X

X X Y Y b

i

i i a ˆ (^) = Y − b ˆ X

Understanding what makes b

• Numerator of b is made of of TWO parts

• Deviations of X from its mean
• Deviations of Y from its mean

• Denominator of b is made up of the deviation

of x from its mean times itself

• Thus b is made of of changes in X times

changes in Y, divided by changes in X squared

• A.K.A “rise over run”

X X

X X Y Y

b

i

i i

Understanding What Makes b

• This corresponds to our intuitive

understanding of the slope of a line

• How much change in Y do we observe for each

change in X?

• We can also see how b is calculated in units of

the dependent variable.

• It is changes in the dependent variable over

changes in the independent variable

X X

X X Y Y

b

i

i i

Let’s Do An Example!

Y X

8 2

2 0

5 1

26 8

14 4

17 5

26 8

Calculating a and b

Y X Y - mean X - mean (X-X)(Y-Y) (X-X)(X-X) 8 2 -6 -2 12 4 2 0 -12 -4 48 16 5 1 -9 -3 27 9 26 8 12 4 48 16 14 4 0 0 0 0 17 5 3 1 3 1 26 8 12 4 48 16

∑ ( X^ i −^ X^ )( Yi −^ Y^ )^ =^186

∑ ( X^ −^ X^ )^2 =^62

b=186/

b=

Calculating a and b

Y X Y - mean X - mean (X-X)(Y-Y) (X-X)(X-X) 8 2 -6 -2 12 4 2 0 -12 -4 48 16 5 1 -9 -3 27 9 26 8 12 4 48 16 14 4 0 0 0 0 17 5 3 1 3 1 26 8 12 4 48 16

a^ ˆ = Y − b ˆ X

Mean of Y = 14

Mean of X = 4

a= 14-3(4)

a= 2

Our regression line is:

Y = 2 + 3X

Let’s Replicate Sigelman on

Presidential Popularity and Incumbent Vote

Presidential Popularity and Incumbent Vote Share

1940-