Simple Linear Regression, Lecture notes of Statistics

FOUNDATIONAL CONCEPTS Model Specification: What is SLR? y = β₀ + β₁x + ε Response vs. predictor variable What does the line represent? (conditional mean) Slope interpretation. Intercept interpretation.

Typology: Lecture notes

2024/2025

Uploaded on 06/26/2026

az-fin
az-fin 🇺🇸

7 documents

1 / 8

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Regression Analysis
Simple Linear Regression & ANOVA
Nicoleta Serban, Ph.D.
Professor
Simple Linear Regression: The
Basics
School of Industrial and Systems Engineering
About This Lesson
Learning Objectives:
Motivate regression analysis with
examples
Examine the fundamentals of
regression analysis
pf3
pf4
pf5
pf8

Partial preview of the text

Download Simple Linear Regression and more Lecture notes Statistics in PDF only on Docsity!

Regression Analysis

Simple Linear Regression & ANOVA

Nicoleta Serban, Ph.D.

Professor

Simple Linear Regression: The

Basics

School of Industrial and Systems Engineering

About This Lesson

Learning Objectives:

  • Motivate regression analysis with examples
  • Examine the fundamentals of regression analysis

Regression – What Is It?

  • Regression modeling is a modeling technique that we use to analyze and estimate the values of a ( response ) variable by using other variables that it's correlated with.
  • Many problems in engineering and science involve exploring the relationships between two or more variables. Regression is a statistical technique that is very useful for these types of problems.

Example 1

A company, which sells medical supplies to hospitals, clinics, and doctor's offices, had considered the effectiveness of a new advertising program. Management wants to know if the advertisement is related to sales. This company intends to increase the sales with an effective advertising program.

The data are recorded for 41 countries, including both developed and developing countries. The data include the following columns. Country Inflation.difference Exchange.rate.change Developed Australia - 1. 2351 - 3. 1870 1 Austria 1. 5508 1. 4781 1 Belgium 1. 0371 0. 0395 1 Canada 0. 0461 - 1. 6416 1 Chile - 18. 4126 - 20. 6329 0

Data Example 2

Ø In 2000 Bush and Gore were the main candidates for President in the U.S. Buchanan, a strongly conservative candidate, was also on the ballot. In the state of Florida , Bush and Gore essentially tied, hence the counts were examined carefully county by county. Ø Palm Beach County exhibited strange results. Even though the people in this county are not conservative, many votes were cast for Buchanan. Examination of the voting ballot revealed that it was easy to mistakenly vote for Buchanan (a conservative candidate) when intending to vote for Gore. We will thus predict whether those who voted for Buchanan were indeed going for a conservative candidate.

Example 3

Variables in Regression

The regression framework is characterized by the following:

  1. We have one particular variable that we are interested in understanding or modelling, such as sales of a particular product, or the stock price of a publicly traded firm. This variable is called the response (dependent) variable , and is usually represented by Y.
  2. We have a set of other variables that we think might be useful in predicting or modelling the response variable (say the price of the product, the competitors’ price, and so on; or the profits, revenues, financial position of the firm, and so on). These are called the predicting or explanatory (independent) variables , and are usually represented by x1, x2, etc.

Variables in Regression

RESPONSE VARIABLE versus PREDICTING VARIABLE? Response Variable: It is a Random Variable. It varies with changes in the predictor/s along with other random changes. Predicting Variable: It is a Fixed Variable. It does not change with the response, but it is set fixed before the response is measured.

Objectives in Regression Analysis

A regression analysis is used for:

1. Prediction of the response variable; 2. Modelling the relationship between the response variable and the explanatory variables; or 3. Testing hypotheses of association relationships. Linear Regression: The basis of what we will be talking about most of this course is the linear model. Virtually all other methods for studying dependence among variables are variations on the idea of linear regression. “All models are wrong, but some are useful. “ George Box “Embrace your data, not your models .” John Tukey

Linear Regression: General Model

Simple linear regression Y = # 0 + # 1 X + ε Multiple linear regression Y = # 0 + # 1 X 1 + # 2 X 2 + ε Polynomial Regression Y = # 0 + # 1 X +# 2 X^2 + ε .

. (^). . . .

.

. (^). . . .

Whether a linear or polynomial model in X, we can estimate the relationship using linear regression.

Linear Regression: General Model

Simple linear regression Y = # 0 + # 1 X + ε Multiple linear regression Y = # 0 + # 1 X 1 + # 2 X 2 + ε Polynomial Regression Y = # 0 + # 1 X +# 2 X^2 + ε