3. Robustifying Least Squares: the rLS filter


{filtX, KG, PreviousPs, clipInd} = 1731 rlsfil(y, mu, Sig, H, F, Q, R, bs, b)
calculates the rLS filter for a (multivariate) time series
{b, ctrl} = 1734 calibrLS(T, Sig, H, F, Q, R, e, N, eps, itmax, aus)
calibrates the rLS filter to a given efficiency loss in the ideal model.


3.1 Derivation

Instead of $ M\Delta y$ we use a huberized version of it, that is

$ H_b(M\Delta y)=M\Delta y \min\{1, \frac{b}{\Vert M\Delta y\Vert}\},$ (4)

so that the correction is just as in the Kalman filter if it is not too large in absolute value, and if it is too large, the direction remains unchanged and it is projected to the ball with radius $ b$ in $ n$ dimensions.

This gives us

$\displaystyle x_{0\vert}$ $\displaystyle =$ $\displaystyle \mu,$ (5)
$\displaystyle x_{t\vert t-1}$ $\displaystyle =$ $\displaystyle F x_{t-1\vert t-1}$ (6)
$\displaystyle x_{t\vert t}$ $\displaystyle =$ $\displaystyle x_{t\vert t-1} + H_b(\hat M_t (y_t-Hx_{t\vert t-1})),$ (7)

where we have not yet specified how to choose $ M$ and $ b$, but (c.f. Ruckdeschel (1999)) We already note at this point that the rLS has preserved the crucial features from of the Kalman filter So in the implementation, we use $ \hat M$.


3.2 Calibration

As to the choice of $ b$ we propose an assurance criterion: How much efficiency in the ideal model relative to the optimal procedure ,i.e. the Kalman filter, am I ready to pay in order to get robustness under deviations from the ideal model? This efficiency loss is quantified as the relative degradation in MSE in the ideal model one obtains if one uses robust versions instead of the classical Kalman filter. Of course the lower the clipping $ b$, the higher the relative efficiency loss, so due to this monoticity we may solve

$ \mathop{\rm {{}E{}}}\nolimits \Vert\Delta x-H_b(\hat M \Delta y)\Vert^2 \stackrel{!}{=}(1+\delta) \mathop{\rm tr}\nolimits \Sigma_{t\vert t}$ (8)

in $ b$, uniquely for a given efficiency loss $ \delta$.

We note here that


3.3 Examples

All examples stem--at least for the state space model specification--from Petr Franek (see Härdle, Klinke, and Müller, 2000, Chapter 10). They have been slightly modified to better demonstrate the features of our methods.


3.3.1 Example 5

As first example we have taken Petr Franek's data kalman.dat together with his specifications for the system matrices and by manipulating observations $ 50$, $ 60$, $ 90$ entered three artificial outliers; the result is saved in kalmanao.dat.


  library("xplore")

  library("plot")

  library("kalman")



  serie = read("kalmanao.dat")

  y = serie[,2]

  mu = 10

  Sig = 0

  H = 1

  F = 1

  Q = 9

  R = 9

  T=dim(y)



  e=0.05

  N=100

  eps=0.01

  itmax=15

  aus=4

The rLS filter is then calibrated to an efficiency loss of $ 5\%$

  ergLS=calibrLS(T,Sig,H,F,Q,R,e,N,eps,itmax,aus)

  b=ergLS.b

Next we filter the data with the Kalman filter and the rLS filter.

  res = kfilter2(y,mu,Sig, H,F,Q,R)

  fx = res.filtX

  res= rlsfil(y,mu,Sig, H,F,Q,R,b)

  frx = res.filtX

The results are then displayed, the classical Kalman filter in red, the rLS filter in green and the observations from kalmanao.dat in blue. Additionally we set flags on the time instances where the rLS filter clips $ \hat M_t\Delta y_t$.

  origy= serie[,1]~serie[,2]

  origy= setmask(origy, "line", "blue", "thin")

  fx = serie[,1]~fx

  fx = setmask(fx, "line", "red", "thin")

  frx = serie[,1]~frx

  frx = setmask(frx, "line", "green", "thin")

  clip=serie[,1]~(res.clipInd)

  clip=paf(clip,clip[,2]==1)

  clip[,2]=0

  setmaskp(clip,4, 3, 4)  



  disp = createdisplay(1,1)

  show(disp,1,1, origy,fx,frx,clip)

  setgopt(disp,1,1, "title", "KalmanData1 + AO's in t=50,60,90")

  setgopt(disp,1,1, "xlabel", "t") 

  setgopt(disp,1,1, "ylabel", "y, rLS, Kalman")

1757 rkalm04.xpl

The graphical result of this example is displayed in Figure 3.

Figure 3: Actual observations $ \hat=$ solid line, classical Kalman filter $ \hat=$ dashed and a bit lighter line, the rLS filter $ \hat=$ dotted line; clipped instances $ \hat=$ circles on top of the graphic.
\includegraphics[scale=0.6]{rlsbsp1.ps}


3.3.2 Example 5

As a second example we took the state-space model underlying Petr Franek's Example 2.


  library("xplore")

  library("plot")

  library("kalman")



  mu = #(20,0)

  Sig = #(0,0)~#(0,0)

  H  = #(0.3,-0.3)~#(1,1)

  F  = #(1,0)~#(1,0)

  R  = #(0,0)~#(0,9)

  mid=#(0,0)

  Qid= #(9,0)~#(0,9)

According to this model we simulated $ 50$ states and observations from a $ 10\%$-AO-contamination using quantlets 1765 epscontnorm and 1768 kemitor2. The contaminating distribution is $ {N}_2\left(\left(\!\!\begin{array}{c} 25\\ 30\end{array}\!\!\right),\left(\!\!\begin{array}{cc} 0.9& 0\\ 0&0.9 \end{array}\!\!\right)\right)$.

  T=50

  e=0.1

           

  mcont=#(25,30)

  Qcont=0.1.*Qid

  AOr=0.1

  randomize(0)

  ErrX = epscontnorm(T,0,mid,R,mcont,Qcont,0) 

  ErrY = epscontnorm(T,AOr,mid,Qid,mcont,Qcont,0)

  sim = kemitor2(T,mu,H,F,(ErrY.X),(ErrX.X))

  y=sim.Y        

  Xact=sim.X

The rLS filter is then calibrated to an efficiency loss of $ 10\%$.

  N=10000

  eps=0.01

  itmax=15

  aus=4



  ergLS=calibrLS(T,Sig,H,F,Qid,R,e,N,eps,itmax,aus)



  b=ergLS.b

The simulated data are filtered by the classical Kalman filter and the rLS filter

  res = kfilter2(y,mu,Sig, H,F,Qid,R)

  fx = res.filtX

  res= rlsfil(y,mu,Sig, H,F,Qid,R,b)

  frx = res.filtX

Next, the filtered values are prepared for graphical output: the classical Kalman filter is to be displayed in red, the rLS filter in green and the actual simulated states in blue.

  i=(1:T)

  Xact1 = i~(Xact[,1])

  Xact1 = setmask(Xact1, "line", "blue", "thin")

  fx1 = i~(fx[,1])

  fx1 = setmask(fx1, "line", "red", "thin")

  frx1= i~(frx[,1])

  frx1 = setmask(frx1, "line", "green", "thin")

Additionally we set green flags on the time instances where the rLS filter clips $ \hat M_t\Delta y_t$ on bottom of the graphics and red ones on top where an AO-outlier occurred.

  ym1=max(vec((Xact[,1])~(fx[,1])~(frx[,1]))) ;top of graphics

  ym2=min(vec((Xact[,1])~(fx[,1])~(frx[,1]))) ;bottom of graphics

  flagInd=i~(ErrY.Ind)

  flagInd=paf(flagInd,flagInd[,2]==1)

  flagInd[,2]=ym1*((ym1>0)*1.1+(ym1<0)*0.9)

  flagclip=i~(res.clipInd)

  flagclip=paf(flagclip,flagclip[,2]==1)

  flagclip[,2]= ym2*((ym2<0)*1.1+(ym2>0)*0.9)

  setmaskp(flagInd,4, 3, 4)  

  setmaskp(flagclip,2, 4, 4)

Finally, all the results of the 1-st coordinate are displayed in the left half of a graphic window.

  disp = createdisplay(1,2)

  show(disp,1,1,Xact1,fx1,frx1,flagInd,flagclip)

  setgopt(disp,1,1, "title", "simulated Model under AO 

                                                 -- 1st coord.")

  setgopt(disp,1,1, "xlabel", "t") 

  setgopt(disp,1,1, "ylabel", "x, rLS, Kalman")

1772 rkalm05.xpl

Then the same is done for the second coordinate, replacing [,1] by [,2] on all instances, and the results are then plotted on the right half of the graphic-window. All the program may be seen in 1777 rkalm05.xpl .

Figure 4: Simulated data according to model from Example 2: actual states $ \hat=$ solid line, the classical Kalman filter $ \hat=$ dashed / a bit lighter, the rLS filter $ \hat=$ dotted line; clipped instances are marked by circles on bottom of the graphic, the AO-instances by circles on top.
\includegraphics[scale=0.6]{rlsbsp2.ps}


3.3.3 Example 6

To show that robustification does not go for free, we have taken Petr Franek's data set kalman2.dat as it was, together with his specification of the state space model. So the data stems from the ideal situation and the classical Kalman filter in this case should do better than the rLS.


  library("xplore")

  library("plot")

  library("kalman")



  serie = read("kalman2.dat")

  y = serie[,2]

  mu = #(0,0)

  Sig = #(0,0)~#(0,0)

  H = #(1,0)'

  F = #(0.5,1)~#(-0.3,0)

  R = #(1,0)~#(0,0)

  Q = 4



  e=0.05

  N=10000

  eps=0.01

  itmax=15

  aus=4

Here the rLS filter is calibrated to an efficiency loss of $ 5\%$.

  ergLS=calibrLS(T,Sig,H,F,Q,R,e,N,eps,itmax,aus)

  b=ergLS.b



  res= kfilter2(y,mu,Sig, H,F,Q,R)

  fx = res.filtX

  fy=(H*fx')'

  res= rlsfil(y,mu,Sig, H,F,Q,R,b)

  frx = res.filtX

  fry=(H*frx')'

As we do not have any information about the actual states in this example, we examine the filtered observations $ y_{t\vert t}$ generated by the Kalman filter and the rLS filter and compare them to the actual observations $ y_{t}$. The filtered observations are simply calculated as $ y_{t\vert t}=Hx_{t\vert t}$.

These filtered observations are then displayed, as usual.


  origy = serie[,1]~serie[,2]

  origy = setmask(origy, "line", "blue", "thin")



  fy = serie[,1]~fy

  fy = setmask(fy, "line", "red", "thin")

  fry = serie[,1]~fry

  fry = setmask(fry, "line", "green", "thin")

  flags = serie[,1]~(res.clipInd)

  flags=paf(flags,flags[,2]==1)

  setmaskp(flags,4, 3, 4)

  

  disp = createdisplay(1,1)

  show(disp,1,1,origy,fy,fry,flags)

  setgopt(disp,1,1, "title", "KalmanData2 in Observation Space")

  setgopt(disp,1,1, "xlabel", "t") 

  setgopt(disp,1,1, "ylabel", "y, y-rLS, y-Kalman")

1786 rkalm06.xpl

The graphical result of this example is displayed in Figure 5.

Figure 5: actual observations $ \hat=$ solid line, the classical Kalman filter dashed and a bit lighter line, the rLS filter $ \hat=$ dotted line; the clipped instances are marked by circles.
\includegraphics[scale=0.6]{rlsbsp3.ps}


3.4 Possible Extensions

Alternatively to using 1794 rlsbnorm and 1797 rlsbnorm1 although the assumption of normality cannot be exactly met, one could use
Simulation of a bundle of paths and then MC-Integration:

One can quite easily replace the routines 1800 rlsbnorm and 1803 rlsbnorm1 by MC-Routines calculating $ \mathop{\rm {{}E{}}}\nolimits \Vert\Delta x-H_b(\hat M \Delta y)\Vert^2$ ob basis of a large sample of $ (\Delta y_t, \Delta x_t)$, that is evolving according to recursions (2), simultaneously to $ x_{t\vert t}$.
Numerical Integration:

Even keeping the normality assumption, in order to speed up the calibration step, one could try and evaluate $ \mathop{\rm {{}E{}}}\nolimits \Vert\Delta x-H_b(\hat M \Delta y)\Vert^2$ for low dimension $ n$ by numerical integration routines instead of using MC-integration.



Method and Data Technologies   MD*TECH Method and Data Technologies
  http://www.mdtech.de  mdtech@mdtech.de