Regression Methods
Jörg Aßmus
June 28, 1999
Simply speaking, a regression problem is a way to determine
a function
describing a functional relation
between a p-dimensional variable
and an output variable Y
where
is a random error term.
There are two different approaches of fitting the function m. In the first
approach, we define m by a finite-dimensional parameter
:
where it is sufficient to estimate the parameter
.
In this case, we are
able to approximate the function at each point
using only the
parameter estimate
of
by
This is called a parametric model. A very useful example is the linear
regression
On the other hand a model is called nonparametric if the dimension p of
the parameter
is infinite. This means that we do not know
anything about the function m. In this case we have to use the data set for
the calculation of the estimated function
at each point
x. We investigate this question in
the Smoothing Methods Tutorial.
Both models can be estimated with XploRe.
A simple way of choosing the appropriate method is given here:
- If we know the functional form of m, we use a parametric
model:
- m is linear, X is one-dimensional,
We use simple linear regression. (
linreg,
gls)
- m is linear, X is p-dimensional,
We use multiple linear regression.
(
linreg,
gls,
linregbs,
linregfs,
linregstep)
- m is nonlinear, e.g.
We use nonlinear regression.
- If we don't know the functional form of m, then we use a
nonparametric model. Nonparametric methods are introduced in
the Smoothing Methods Tutorial.
The nonlinear methods are more general than the linear methods. This means that
we can use the nonlinear regression for estimating linear models, but this
cannot be recommended in general. In the same way, the nonparametric models are
more general than the parametric ones.
In the following sections, the libraries stats
containing the quantlets for the regression and
graphic
containing the plot quantlets
will be used. We should load them before we continue:
library("graphic") ; reads the library graphic
library("stats") ; reads the library stats