Real-Time Recurrent Learning (RTRL) Algorithm

From GM-RKB
Jump to navigation Jump to search

A Real-Time Recurrent Learning (RTRL) Algorithm is a Gradient Descent Algorithm that is an online learning algorithm for training RNNs.



References

2021

2020

$\nabla_{\theta} \mathcal{L}=\displaystyle \sum_{t=1}^{T} \dfrac{\partial \mathcal{L}_{t}}{\partial h_{t}} \dfrac{\partial h_{t}}{\partial \theta}=\sum_{t=1}^{T} \dfrac{\partial \mathcal{L}_{t}}{\partial h_{t}}\left(\dfrac{\partial h_{t}}{\partial \theta_{t}}+\dfrac{\partial h_{t}}{\partial h_{t-1}} \dfrac{\partial h_{t-1}}{\partial \theta}\right)$

(2)
This can be viewed as an iterative algorithm, updating $\frac{\partial h_t}{\partial \theta}$ from the intermediate quantity $\frac{\partial h_{t−1}}{\partial\theta}$. To simplify equation 2 we introduce the following notation: have $J_{t}:=\frac{\partial h_{t}}{\partial \theta}$, $I_{t}:=\frac{\partial h_{t}}{\partial \theta_{t}}$ and $D_{t}:=\frac{\partial h_{t}}{\partial h_{t-1}}$. $J$ stands for “Jacobian”, $I$ for “immediate Jacobian”, and $D$ for “dynamics”. We sometimes refer to $J$ as the “influence matrix”. The recursion can be rewritten $J_t = I_t + D_tJ_{t−1}$.

2019

2018

2017

2013

1989

  • (Williams & Zipser, 1989) ⇒ Ronald J. Williams, and David Zipser (1989). “A Learning Algorithm for Continually Running Fully Recurrent Neural Networks". In: Neural Computation 1(2): 270-280.

1987