Multivariate Regression is a method used to measure the degree at which more than one independent variable (predictors) and more than one dependent variable (responses), are linearly related. The method is broadly used to predict the behavior of the response variables associated to changes in the predictor variables, once a desired degree of relation has been established.
Exploratory Question: Can a supermarket owner maintain stock of water, ice cream, frozen foods, canned foods and meat as a function of temperature, tornado chance and gas price during tornado season in June?
From this question, several obvious assumptions can be drawn: If it is too hot, ice cream sales increase; If a tornado hits, water and canned foods sales increase while ice cream, frozen foods and meat will decrease; If gas prices increase, prices on all goods will increase. A mathematical model, based on multivariate regression analysis will address this and other more complicated questions.
The Simple Regression model, relates one predictor and one response.
The fitted model (fitted to the given data) is as follows:
The estimated parameters are and , such that and are the sample averages.
Note: In most applications, it is assumed that error terms are iid . In general the error terms are not assumed to follow a particular distribution, they are assumed to be , and for , expected value, variance and covariance.
The Multiple Regression model, relates more than one predictor and one response.
The Multivariate Regression model, relates more than one predictor and more than one response.
Let be the response matrix, be an matrix such that all entries of the first column are , and predictors. Let be an matrix of fixed parameters, be an matrix such that (multivariate normally distributed with covariance matrix ). The model is as follows:
In detail notation we have:
This estimator minimizes .
The unbiased estimator for , denoted :
The fitted (prediction) model given by is as follows:
With predicted error .
Sample Covariance and
The matrix of sample covariance, , is given by a block matrix such that , , and , and has the following form:
A measure on the association of the variables of the model will be denoted , with a range between zero and one. This measure, , is the largest eigenvalue of the following matrix: