3. Nonlinear Regression

A more difficult type of parametric models is the nonlinear regression model. Here we cannot find just one nice and easy algorithm to solve all problems, as in the linear case. We consider the general parametric model

\begin{displaymath}Y=m_\beta(X)+\varepsilon,\end{displaymath}

where $m_\beta$ is a nonlinear function depending on the parameter vector
[3] $\beta=(\beta_0,\ldots,\beta_p)$ and $\varepsilon$ a random error term. Examples can be

\begin{eqnarray*}

Y&=&\beta X^2+\varepsilon\,\,\textrm{or}\\

Y&=&\beta_1X_1^2+\beta_2X_2^4\,\,\textrm{or}\\

Y&=&\beta_1X^{\beta_2}\,.

\end{eqnarray*}


Considering these three examples we can distinguish two types of nonlinear regression. The first two examples can be referred to as a curvilinear regression model leading to a simple linear regression model or multiple linear regression model by a transformation of the variables. The third can be called a ``real nonlinear model''. This will not be considered here.

We will face the general curvilinear model

\begin{displaymath}Y=\beta_0+\sum\limits_{i=1}^p\beta_if_i(X_i)+\varepsilon\end{displaymath}

where at least one of the functions fi is assumed to be nonlinear. Using transformed variables $\widetilde X_i=f_i(X_i)$ we simply get the linear model

\begin{displaymath}Y=\beta_0+\sum\limits_{i=1}^p\beta_i\widetilde X_i+\varepsilon\,.\end{displaymath}

That means that we only have to transform the data set in the same way and estimate the parameter $\beta$ by the linear regression. As an example, we consider a model leading to the a simple linear regression. It is stored in 969regr4.xpl. The model is given as

\begin{displaymath}Y=2X^2+\varepsilon,\end{displaymath}

where $\varepsilon$ is normal distributed. As in the previous sections, we generate a data set accordingly:

  randomize(0)         ; sets a seed for the random generator

  x=2*uniform(20)-1    ; generates 20 on [-1,1] uniform

                       ;    distributed values

  eps=normal(20)       ; generates 20 standard normal distributed

                       ;    values

  y=2*x^2+eps/2        ; creates y

Next we transform the data $\widetilde X=X^2$ and compute the simple linear regression using the quantlet 972 linreg.


  xtrans=x^2           ; transforms x into the variable xtrans

  {beta,bse,bstan,bpval}=linreg(xtrans,y)

                       ; computes the linear regression

As in the linear case, we get the ANOVA and parameter tables.

  A  N  O  V  A          SS      df     MSS     F-test   P-value

  ______________________________________________________________

  Regression            7.781     1     7.781    30.880   0.0000

  Residuals             4.535    18     0.252

  Total Variation      12.316    19     0.648



  Multiple R      = 0.79483

  R^2             = 0.63175

  Adjusted R^2    = 0.61130

  Standard Error  = 0.50196



  PARAMETERS    Beta       SE       StandB      t-test  P-value

  _____________________________________________________________

  b[ 0,]=    -0.1143     0.1785    -0.0000      -0.640   0.7350

  b[ 1,]=     1.7433     0.3137     0.7948       5.557   0.0000

Both tables are interpreted as in the previous sections. The parameter table shows the estimates for $\beta_0$ and $\beta_1$. From its high p-value, we see that the true value of the constant $\beta_0$ is estimated to be zero. The estimated regression function is

\begin{displaymath}\widehat Y(x)= 1.7433 \widetilde x = 1.7433 x^2\,.\end{displaymath}

Now we want to visualize this result.


  data=sort(x~y)             ; creates the graphical object for

                             ;    the data

  setmaskp(data,1,11,8)      ; sets the options for the graphical

                             ;    object data

  rx=#(-10:10)/10            ; creates the vector (-1,-0.9,...,1)

  yq=(beta[2]*rx^2)          ; creates vector with the regressed

                             ;    values

  rdata=sort(rx~yq)          ; creates the graphical object for

                             ;    the regressed values

  rdata=setmask(rdata,"reset","line","red","medium")

                             ; sets the options for the graphical

                             ;    object rdata

  rf=sort(rx~2*rx^2)         ; creates the graphical object for

                             ;    the original function

  rf=setmask(rf,"reset","line","green","medium")

                             ; sets the options for the graphical

                             ;    object rf

  nlplot=createdisplay(1,1)  ; creates the display nlplot

  show(nlplot,1,1,data,rdata,rf) ;shows the display nlplot

  setgopt(nlplot,1,1,"title","Nonlinear Regression")

                             ; sets the title of the display

Figure: Nonlinear regression of $Y=2X^2+\varepsilon $ using the variable transformation. 976 regr4.xpl
\includegraphics[scale=0.6]{linreg4.ps}

Looking at this plot, we see that the transformation of the variables leads to a good reproduction of the shape of the true function, even when the errors are in the same range as the function itself. But we should remember that we have to know the functions fi.

We don not want to consider the case leading to a multiple linear regression because the principle is exactly the same as with our example. We just have to use the methods of Section 2.



Method and Data Technologies   MD*TECH Method and Data Technologies
  http://www.mdtech.de  mdtech@mdtech.de