Comparison of RLS to RSR
 Comparison of RLS to RSR All System Identification Articles All Parameter Identification Articles In order to prevent spam, users must register before they can edit or create articles.

## 1 Comparison of Online Recursive Least Squares to Online Recursive Square Root

### 1.1 Conversion of Online Recursive Least Squares to formulation presented in Hsia

The starting update equations are[1]

 $LaTeX: \hat{\theta}\left(m+1\right)=\hat{\theta}\left(m\right)+\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right) \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ]$ Eqn. 1

and

 $LaTeX: P\left(m+1\right)=P\left(m\right)-\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right)x^{T}\left(m+1\right)p\left(m\right),$ Eqn. 2

where

 $LaTeX: \gamma\left(m+1\right)=\frac{1}{1+x^{T}\left(m+1\right)P\left(m\right)x\left(m+1\right)}$ Eqn. 3

From the RLS algorithm we have

 $LaTeX: \hat{\theta}_{p+1}^{(i+1)}=\hat{\theta}_{p}^{(i)}+K_{p+1}^{(i+1)}e_{p+1}^{(i+1)},$ Eqn. 4

where

 $LaTeX: e_{p+1}^{(i+1)}=\hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{T(i+1)}\hat{\theta}_{p}^{(i)},$ Eqn. 5

 $LaTeX: \hat{\theta}_{p+1}^{(i+1)}=\hat{\theta}_{p}^{(i)}+K_{p+1}^{(i+1)} \left [ \hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{T(i+1)}\hat{\theta}_{p}^{(i)} \right ].$ Eqn. 6

Comparing Eqns. 1 and 6 leads to the following conclusions

 $LaTeX: \hat{\theta}_{p}^{(i)}=\hat{\theta}\left(m\right),$

 $LaTeX: e_{p+1}^{(i+1)}= \left [ \hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{(T(i+1)}\hat{\theta}_{p}^{(i)} \right ]$$LaTeX: \rArr$$LaTeX: \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ]$

 $LaTeX: \hat{y}_{p+1}^{(i+1)}=y\left(m+1\right)$

 $LaTeX: \phi_{p+1}^{T(i+1)}=x^{T}\left(m+1\right)$

With these substitutions it is clear that the Kalman gain is

 $LaTeX: K_{p+1}^{(i+1)}=\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right).$

From the RLS algorithm we have

 $LaTeX: K_{p+1}^{(i+1)}=\frac{P_{p}^{(i)}\phi_{p+1}^{(i+1)}}{\left(\frac{1}{w_{p+1}}\right)+\phi_{p+1}^{T(i+1)}P_{p}^{(i)}\phi_{p+1}^{(i+1)}}$

if $LaTeX: \lambda_{p+1}=1$. If $LaTeX: w_{p+1}=1$ for all iteration, then

 $LaTeX: K_{p+1}^{(i+1)}=\frac{P_{p}^{(i)}\phi_{p+1}^{(i+1)}}{1+\phi_{p+1}^{T(i+1)}P_{p}^{(i)}\phi_{p+1}^{(i+1)}}$

After making the appropriate substitutions for $LaTeX: \phi_{p+1}^{(i+1)}$, then $LaTeX: K_{p+1}^{(i+1)}$ becomes

 $LaTeX: K_{p+1}^{(i+1)}=\frac{P_{p}^{(i)}x\left(m+1\right)}{1+x^{T}\left(m+1\right)P_{p}^{(i)}x\left(m+1\right)}.$

If $LaTeX: P_{p+1}^{(i)}=P\left(m\right)$, then

 $LaTeX: K_{p+1}^{(i+1)}=\frac{P\left(m\right)x\left(m+1\right)}{1+x^{T}\left(m+1\right)P\left(m\right)x\left(m+1\right)}=\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right)$

In order to double check the $LaTeX: P_{p+1}^{(i)}=P\left(m\right)$ result we start with the update equation for $LaTeX: P_{p}^{(i+1)}$ from the RLS algorithm

 $LaTeX: P_{p}^{(i+1)}=P_{p}^{(i)}-K_{p+1}^{(i+1)}\left(P_{p}^{(i)}\phi_{p+1}^{(i+1)}\right)^{T}.$

Making the correct substitutions we get

 $LaTeX: P\left(m+1\right)=P\left(m\right)-\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right) \left [ P\left(m\right)x\left(m+1\right) \right ]^{T}.$ Eqn. 7

This becomes

 $LaTeX: P\left(m+1\right)=P\left(m\right)-\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right)x^{T}\left(m+1\right)P^{T}\left(m\right).$

which is equal to Eqn. 7 is $LaTeX: P\left(m\right)$ is symmetric. Since $LaTeX: P\left(m\right)$ is originally defined as[2]

 $LaTeX: P\left(m\right)=\left(X_{m}^{T}X_{m}\right)^{-1},$

it must be symmetric.

### 1.2 Conversion of Online Recursive Square Root to formulation presented in Hsia[3]

The RSR update equations are[4]

 $LaTeX: \hat{\theta}_{i}\left(k\right)=\hat{\theta}_{i}\left(k-1\right)+S_{i}\left(k-1\right)f_{i}\left(k\right)g_{i}\left(k\right)\xi_{i}\left(k\right),$ Eqn. 8

where

 $LaTeX: \xi\left(k\right)=x_{di}\left(k\right)-\hat{\theta}_{i}^{T}\left(k-1\right)\phi\left(k\right),$ Eqn. 9

 $LaTeX: f_{i}\left(k\right)=S_{i}^{T}\left(k-1\right)\phi\left(k\right),$ Eqn. 10

 $LaTeX: g_{i}\left(k\right)=\left[f_{i}^{T}\left(k\right)f_{i}\left(k\right)+1\right]^{-1},$ Eqn. 11

 $LaTeX: S_{i}\left(k\right)=\frac{S_{i}\left(k-1\right) \left [ I_{n+m}-g_{i}\left(k\right)\alpha_{i}\left(k\right)f_{i}\left(k\right)f_{i}^{T}\left(k\right) \right ] }{\sqrt{\beta_{i}\left(k\right)}},$ Eqn. 12

for $LaTeX: trace\left [ S_{i}\left(k-1\right)\right ] >c$

and

 $LaTeX: S_{i}\left(k\right)=S_{i}\left(k-1\right)\left [ I_{n+m}-g_{i}\left(k\right)\alpha_{i}\left(k\right)f_{i}\left(k\right)f_{i}^{T}\left(k\right)\right ],$ Eqn. 13

for $LaTeX: trace\left [ S_{i}\left(k-1\right)\right ] >c$

and

 $LaTeX: \alpha_{i}\left(k\right)=\frac{1}{1+\sqrt{g_{i}\left(k\right)}},$ Eqn. 14

and

 $LaTeX: \beta_{i}\left(k\right)=\left | 1-\frac{\left | \xi_{i}\left(k\right) \right | \times \sqrt{g_{i}\left(k\right)}}{\sqrt{\Sigma_{i}}} \right |.$ Eqn. 15

Just as the RLS algorithm was linked to the algorithm derived in Hsia[5] the RSR algorithm from Zhao[6] can also be shown to be equivalent be the correct substitutions. Again, we start with the error or innovation sequence defined as

 $LaTeX: e_{p+1}^{(i+1)}= \left [ \hat{y}_{p+1}^{(i+1)}-\phi_{p+1}^{(T(i+1)}\hat{\theta}_{p}^{(i)} \right ]$$LaTeX: \rArr$$LaTeX: \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ]$

for the RLS algorithm and

 $LaTeX: \xi_{i}\left(k\right)=x_{di}\left(k\right)-\hat{\theta}_{i}^{T}\left(k-1\right)\phi\left(k\right)$$LaTeX: \rArr$$LaTeX: \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ] ^{T}$

for the RSR algorithm. Teh resulting equalities fall out

 $LaTeX: x_{di}\left(k\right)=y^{T}\left(m+1\right),$

 $LaTeX: \hat{\theta}_{i}\left(k-1\right)=\hat{\theta}\left(m\right),$ Eqn. 16

and

 $LaTeX: \phi\left(k\right)=x\left(m+1\right)$ Eqn. 17

. Using Eqn. 17 in Eqn. 10

 $LaTeX: f_{i}\left(k\right)=S_{i}^{T}\left(k-1\right)x\left(m+1\right)$

and Eqn. 11 becomes

 $LaTeX: g_{i}\left(k\right)=\left [ f_{i}^{T}\left(k\right)f_{i}\left(k\right)+1 \right ] ^{-1}=\left [ x^{T}\left(m+1\right)S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right)x\left(m+1\right)+1 \right ] ^{-1}.$

If $LaTeX: g_{i}\left(k\right)$ above is compared to $LaTeX: \gamma\left(m+1\right)$. from Eqn. 3, then

 $LaTeX: g_{i}\left(k\right)=\gamma\left(m+1\right)$

with the following equality

 $LaTeX: P\left(m\right)=S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right).$ Eqn. 18

Then Eqn. 1 becomes

 $LaTeX: \hat{\theta}_{i}\left(k\right)=\hat{\theta}_{i}\left(k-1\right)+S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right)x\left(m+1\right) \left [ x^{T}\left(m+1\right)S_{i}\left(k-1\right)S_{i}^{T}\left(k-1\right)x\left(m+1\right)+1 \right ] ^{-1} \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ].$

Use Eqn. 18 and get

 $LaTeX: \hat{\theta}_{i}\left(k\right)=\hat{\theta}_{i}\left(k-1\right)+P\left(m\right)x\left(m+1\right) \left [ x^{T}\left(m+1\right)P\left(m\right)x\left(m+1\right)+1 \right ] ^{-1} \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ].$

Substitute in $LaTeX: \gamma\left(m+1\right)$ from Eqn. 3 and $LaTeX: \theta\left(m+1\right)$ from Eqn. 16 to get

 $LaTeX: \hat{\theta}_{i}\left(m+1\right)=\hat{\theta}_{i}\left(m\right)+\gamma\left(m+1\right)P\left(m\right)x\left(m+1\right) \left [ y\left(m+1\right)-x^{T}\left(m+1\right)\hat{\theta}\left(m\right) \right ].$

Note that $LaTeX: \gamma\left(m+1\right)$ is a scalar.

## 3 References

• Spradlin, Gabriel T. '"AN EXPLORATION OF PARAMETER IDENTIFICATION TECHNIQUES: CMG TEMPERATURE PREDICTION THEORY AND RESULTS." Master's Thesis, University of Houston, Houston, TX December 2005.
• Zhao, X. M., Shieh, L. S., Sunkel, J. W., and Yuan, Z. Z., "Self-tuning Control of Attitude and Momentum Management of the Space Station," AIAA J. of Guidance, Control, and Dynamics, Vol. 15, No. 1, 1992, pp. 17-27.
• Hsia, T. C., "System Identification (Least-Squares Methods)," 1977 D. C. Heath and Company, Lexington, MA. ISBN 0669996300

1. Hsia
2. Hsia