Elman Neural Network for Data Prediction - Research on Power Load Forecasting Model

The Elman neural network was proposed by Elman in 1990. This model enhances the standard feedforward network by adding a context layer to the hidden layer, which acts as a one-step delay operator to incorporate memory functionality. This architecture enables the system to adapt to time-varying characteristics and directly capture dynamic process system properties. In code implementation, this memory mechanism is typically achieved through recurrent connections that maintain hidden state information across time steps.

The Elman recurrent neural network generally consists of four layers: input layer, hidden layer (intermediate layer), context layer, and output layer. In this model, the input layer receives external input signals, the hidden layer processes these inputs, and the context layer transfers the previous time step's output values to the current input layer to implement memory functionality. Finally, the output layer generates the final prediction based on signals processed through the hidden and context layers. From a programming perspective, the context layer can be implemented using state variables that store historical hidden layer activations, often through weight matrices connecting hidden layer outputs back to the hidden layer inputs.

As shown in the diagram below, the connections between input, hidden, and output layers resemble those in feedforward networks. The input layer units primarily handle signal transmission, while output layer units perform linear weighting operations. Hidden layer units can employ either linear or nonlinear activation functions (such as sigmoid or tanh functions) to process input signals. The context layer, also referred to as the state layer or contextual layer, serves to memorize the previous time step's output values from hidden layer units and feedback them to the input layer, effectively functioning as a one-step delay operator. Algorithmically, this creates a simple recurrent network where the context layer's values are typically computed as: context(t) = hidden(t-1), creating a delayed feedback loop.

By incorporating the context layer, the Elman neural network adds memory capability to the standard feedforward architecture, allowing the system to adapt to time-varying characteristics and directly reflect dynamic process system behavior. This makes it particularly suitable for time series forecasting applications like power load prediction, where implementation often involves unfolding the network through time and using backpropagation through time (BPTT) for training.