Comparison of RLS to RSR

From ControlTheoryPro.com

Jump to: navigation, search
Symbol.gif
Comparison of RLS to RSR
Green carrot left.gif
All System Identification Articles All Parameter Identification Articles
Green carrot.jpg
In order to prevent spam, users must register before they can edit or create articles.


1 Comparison of Online Recursive Least Squares to Online Recursive Square Root

If you wish to skip to the conclusion click here.

1.1 Conversion of Online Recursive Least Squares to formulation presented in Hsia

The starting update equations are[1]

LaTeX: \hat{\theta}\left(m+1\right)=\hat{\theta}\left(m\right)+\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right) \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ] Eqn. 1


and

LaTeX: P\left(m+1\right)=P\left(m\right)-\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right)x^{T}\left(m+1\right)p\left(m\right), Eqn. 2


where

LaTeX: \gamma\left(m+1\right)=\frac{1}{1+x^{T}\left(m+1\right)P\left(m\right)x\left(m+1\right)} Eqn. 3



From the RLS algorithm we have

LaTeX: \hat{\theta}_{p+1}^{(i+1)}=\hat{\theta}_{p}^{(i)}+K_{p+1}^{(i+1)}e_{p+1}^{(i+1)}, Eqn. 4


where

LaTeX: e_{p+1}^{(i+1)}=\hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{T(i+1)}\hat{\theta}_{p}^{(i)}, Eqn. 5


leading to

LaTeX: \hat{\theta}_{p+1}^{(i+1)}=\hat{\theta}_{p}^{(i)}+K_{p+1}^{(i+1)} \left [ \hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{T(i+1)}\hat{\theta}_{p}^{(i)} \right ]. Eqn. 6



Comparing Eqns. 1 and 6 leads to the following conclusions

LaTeX: \hat{\theta}_{p}^{(i)}=\hat{\theta}\left(m\right),



LaTeX: e_{p+1}^{(i+1)}= \left [ \hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{(T(i+1)}\hat{\theta}_{p}^{(i)} \right ]LaTeX: \rArrLaTeX:  \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ]



LaTeX: \hat{y}_{p+1}^{(i+1)}=y\left(m+1\right)



LaTeX: \phi_{p+1}^{T(i+1)}=x^{T}\left(m+1\right)


With these substitutions it is clear that the Kalman gain is

LaTeX: K_{p+1}^{(i+1)}=\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right).



From the RLS algorithm we have

LaTeX: K_{p+1}^{(i+1)}=\frac{P_{p}^{(i)}\phi_{p+1}^{(i+1)}}{\left(\frac{1}{w_{p+1}}\right)+\phi_{p+1}^{T(i+1)}P_{p}^{(i)}\phi_{p+1}^{(i+1)}}


if LaTeX: \lambda_{p+1}=1. If LaTeX: w_{p+1}=1 for all iteration, then

LaTeX: K_{p+1}^{(i+1)}=\frac{P_{p}^{(i)}\phi_{p+1}^{(i+1)}}{1+\phi_{p+1}^{T(i+1)}P_{p}^{(i)}\phi_{p+1}^{(i+1)}}


After making the appropriate substitutions for LaTeX: \phi_{p+1}^{(i+1)}, then LaTeX: K_{p+1}^{(i+1)} becomes

LaTeX: K_{p+1}^{(i+1)}=\frac{P_{p}^{(i)}x\left(m+1\right)}{1+x^{T}\left(m+1\right)P_{p}^{(i)}x\left(m+1\right)}.


If LaTeX: P_{p+1}^{(i)}=P\left(m\right), then

LaTeX: K_{p+1}^{(i+1)}=\frac{P\left(m\right)x\left(m+1\right)}{1+x^{T}\left(m+1\right)P\left(m\right)x\left(m+1\right)}=\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right)


In order to double check the LaTeX: P_{p+1}^{(i)}=P\left(m\right) result we start with the update equation for LaTeX: P_{p}^{(i+1)} from the RLS algorithm

LaTeX: P_{p}^{(i+1)}=P_{p}^{(i)}-K_{p+1}^{(i+1)}\left(P_{p}^{(i)}\phi_{p+1}^{(i+1)}\right)^{T}.


Making the correct substitutions we get

LaTeX: P\left(m+1\right)=P\left(m\right)-\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right) \left [ P\left(m\right)x\left(m+1\right) \right ]^{T}. Eqn. 7


This becomes

LaTeX: P\left(m+1\right)=P\left(m\right)-\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right)x^{T}\left(m+1\right)P^{T}\left(m\right).


which is equal to Eqn. 7 is LaTeX: P\left(m\right) is symmetric. Since LaTeX: P\left(m\right) is originally defined as[2]

LaTeX: P\left(m\right)=\left(X_{m}^{T}X_{m}\right)^{-1},


it must be symmetric.

1.2 Conversion of Online Recursive Square Root to formulation presented in Hsia[3]

The RSR update equations are[4]

LaTeX: \hat{\theta}_{i}\left(k\right)=\hat{\theta}_{i}\left(k-1\right)+S_{i}\left(k-1\right)f_{i}\left(k\right)g_{i}\left(k\right)\xi_{i}\left(k\right), Eqn. 8


where

LaTeX: \xi\left(k\right)=x_{di}\left(k\right)-\hat{\theta}_{i}^{T}\left(k-1\right)\phi\left(k\right), Eqn. 9



LaTeX: f_{i}\left(k\right)=S_{i}^{T}\left(k-1\right)\phi\left(k\right), Eqn. 10



LaTeX: g_{i}\left(k\right)=\left[f_{i}^{T}\left(k\right)f_{i}\left(k\right)+1\right]^{-1}, Eqn. 11



LaTeX: S_{i}\left(k\right)=\frac{S_{i}\left(k-1\right) \left [ I_{n+m}-g_{i}\left(k\right)\alpha_{i}\left(k\right)f_{i}\left(k\right)f_{i}^{T}\left(k\right) \right ] }{\sqrt{\beta_{i}\left(k\right)}}, Eqn. 12


for LaTeX: trace\left [ S_{i}\left(k-1\right)\right ] >c

and

LaTeX: S_{i}\left(k\right)=S_{i}\left(k-1\right)\left [ I_{n+m}-g_{i}\left(k\right)\alpha_{i}\left(k\right)f_{i}\left(k\right)f_{i}^{T}\left(k\right)\right ], Eqn. 13


for LaTeX: trace\left [ S_{i}\left(k-1\right)\right ] >c

and

LaTeX: \alpha_{i}\left(k\right)=\frac{1}{1+\sqrt{g_{i}\left(k\right)}}, Eqn. 14


and

LaTeX: \beta_{i}\left(k\right)=\left | 1-\frac{\left | \xi_{i}\left(k\right) \right | \times \sqrt{g_{i}\left(k\right)}}{\sqrt{\Sigma_{i}}} \right |. Eqn. 15



Just as the RLS algorithm was linked to the algorithm derived in Hsia[5] the RSR algorithm from Zhao[6] can also be shown to be equivalent be the correct substitutions. Again, we start with the error or innovation sequence defined as

LaTeX: e_{p+1}^{(i+1)}= \left [ \hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{(T(i+1)}\hat{\theta}_{p}^{(i)} \right ]LaTeX: \rArrLaTeX:  \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ]


for the RLS algorithm and

LaTeX: \xi_{i}\left(k\right)=x_{di}\left(k\right)-\hat{\theta}_{i}^{T}\left(k-1\right)\phi\left(k\right)LaTeX: \rArrLaTeX: \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ] ^{T}


for the RSR algorithm. Teh resulting equalities fall out

LaTeX: x_{di}\left(k\right)=y^{T}\left(m+1\right),



LaTeX: \hat{\theta}_{i}\left(k-1\right)=\hat{\theta}\left(m\right), Eqn. 16


and

LaTeX: \phi\left(k\right)=x\left(m+1\right) Eqn. 17


. Using Eqn. 17 in Eqn. 10

LaTeX: f_{i}\left(k\right)=S_{i}^{T}\left(k-1\right)x\left(m+1\right)


and Eqn. 11 becomes

LaTeX: g_{i}\left(k\right)=\left [ f_{i}^{T}\left(k\right)f_{i}\left(k\right)+1 \right ] ^{-1}=\left [ x^{T}\left(m+1\right)S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right)x\left(m+1\right)+1 \right ] ^{-1}.


If LaTeX: g_{i}\left(k\right) above is compared to LaTeX: \gamma\left(m+1\right). from Eqn. 3, then

LaTeX: g_{i}\left(k\right)=\gamma\left(m+1\right)


with the following equality

LaTeX: P\left(m\right)=S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right). Eqn. 18


Then Eqn. 1 becomes

LaTeX: \hat{\theta}_{i}\left(k\right)=\hat{\theta}_{i}\left(k-1\right)+S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right)x\left(m+1\right) \left [ x^{T}\left(m+1\right)S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right)x\left(m+1\right)+1 \right ] ^{-1} \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ].


Use Eqn. 18 and get

LaTeX: \hat{\theta}_{i}\left(k\right)=\hat{\theta}_{i}\left(k-1\right)+P\left(m\right)x\left(m+1\right) \left [ x^{T}\left(m+1\right)P\left(m\right)x\left(m+1\right)+1 \right ] ^{-1} \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ].


Substitute in LaTeX: \gamma\left(m+1\right) from Eqn. 3 and LaTeX: \theta\left(m+1\right) from Eqn. 16 to get

LaTeX: \hat{\theta}_{i}\left(m+1\right)=\hat{\theta}_{i}\left(m\right)+\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right) \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ].


Note that LaTeX: \gamma\left(m+1\right) is a scalar.

2 Conclusion

TODO

TODO
Add conclusion...


3 References

  • Spradlin, Gabriel T. '"AN EXPLORATION OF PARAMETER IDENTIFICATION TECHNIQUES: CMG TEMPERATURE PREDICTION THEORY AND RESULTS." Master's Thesis, University of Houston, Houston, TX December 2005.
  • Zhao, X. M., Shieh, L. S., Sunkel, J. W., and Yuan, Z. Z., "Self-tuning Control of Attitude and Momentum Management of the Space Station," AIAA J. of Guidance, Control, and Dynamics, Vol. 15, No. 1, 1992, pp. 17-27.
  • Hsia, T. C., "System Identification (Least-Squares Methods)," 1977 D. C. Heath and Company, Lexington, MA. ISBN 0669996300

3.1 Notes

  1. Hsia
  2. Hsia
  3. Spradlin, pp. 196-198
  4. Zhao et all
  5. Hsia
  6. Zhao et all