SIMPLE LINEAR
REGRESSION
Assumed Linear Model:
Theoretical basis for
selection of dependent and independent variables
1. Dependent
variable is the variable that we desire to forecast (predict)
2. Independent or
predictor variable(s) are assumed to have a one-way causal effect on the
dependent variable.
Criteria for selection of
independent variables:
1. There is a sound theoretical basis for including the predictor
variable in the model.
2. There is an
adequate data for all of the variables of interest. Rule of thumb: there
should be at least 5 observations for every independent variable used in the
model to guard against “overfitting.”
3. There should
be future projections of the independent variables or a lead-lag relationship
should be established, so that the dependent variable is predicted for “known”
independent variables.
4. The historical
relationship is likely to continue into the future.
Steps in Estimating the
Relationship
1. Always begin
with a scatter diagram with the dependent variable on the y-axis and the
independent variable on the x-axis.
2. Examine the
correlation coefficient matrix among the dependent and independent
variable(s).
3.
Estimate the least squares
regression line.
Linear Regression Model
1.
Statistical formulation
(what determines the values of the coefficients?)
2.
Statistical estimation (how
do we estimate the regression coefficients?)
3.
Properties of the
least-squares model (assumptions that model should satisfy)
a. No specification error (linear, relevant independent variables
included)
b. No measurement error
c. Error term (random, constant variance, no autocorrelation, uncorrelated
with independent variables, normal distribution.)
` 4. Estimated
regression equation versus true regression equation.
a. Parameter estimation
b. Statistical validation (hypothesis testing)
5. Model performance
c. Standard error of the estimate
d. Analysis of variance