Abstract

ON THE TRAINING OF RECURRENT NEURAL NETWORKS MODEL

Hamzah Abubakari,*1 Sagir Abdu Masanawa2 and Surajo Yusuf3

---

This paper presents an elementary theory of the basic backpropagation neural network architecture, covering the ares of architectural design, performance measurement, function approximation capability and learning. The basic equations for backpropagation through time, and applications to ares such as pattern recontion involving dynamic system are presented. In order to demonstrate the feasibility of expanding feedforward neural network to reccurnet networks for training purposes is presented. This paper provides detailed description and necessary derivations for the BackPropagation Through Time (BPTT) algorithm. BPTT is often used to learn recurrent neural networks (RNN). Contrary to feed-forwardneural networks, the RNN is characterized by the ability of encoding longer past information, thus very suitable for sequential models. The BPTT extends the ordinary BP algorithm to suit the recurrent neural architecture. Finally, it explains further extensions of this approach to tackle models other than neural networks, systems containing simultaneous equations or actual recurrent networks, and other practical issues resulting from this method. The chain rule for ordered derivatives-the theorem which underlies backpropagation is also prensented. Keywords: Artificial neural network, recurrent neural networks, Backpropagation, Backpropagation through Time and backpropagation algorithm.

PDF