A Novel Short Receptive Field based Dilated Causal Convolutional Network Integrated with Bidirectional LSTM for Short-Term Load Forecasting

Authors	Umar Javed, Khalid Ijaz, Muhammad Jawad, Ikramullah Khosa, Ejaz Ahmad Ansari, Khurram Shabih Zaidi, Muhammad Nadeem Rafiq, and Noman Shabbir
Title	A Novel Short Receptive Field based Dilated Causal Convolutional Network Integrated with Bidirectional LSTM for Short-Term Load Forecasting
Publication	Expert Systems with Applications
Volume	205
Issue	x
Pages	x
Year	2022
DOI	https://doi.org/10.1016/j.eswa.2022.117689

Introduction

Background

• Electrical load forecasting is crucial for power systems for economic operations, such as unit commitment and maintenance and demand schedule for power generating units.
• An accurate forecast helps distribution system operators to avoid blackouts and assists them in planning diversified generations with stability.
• Inaccurate load forecast impacts generation planning and also hampers the protection and security of electrical power systems.
• STLF (Short-Term Load Forecasting) can play a crucial role in structuring economic, secure, and reliable operating strategies for distribution electrical utilities.
• The motivation of the research paper is to empower electric utilities with a state-of-the-art and improved hybrid DNN (Deep Neural Network) methodology for the optimal solution of the STLF problem

Previous Research

• High nonlinearity in electrical load patterns is a challenging problem to construct an accurate STLF model.
• Research works in the past two decades have been presented on STLF with a prediction horizon from one hour to a few weeks.
• Statistical regression methods models lack the capability to capture temporal variations and non-linear electrical load patterns.
• Performance of statistical methods is enhanced using PCA (Principal Component Analysis) but may result in the loss of key features if the coefficients of covariance matrix are not adjusted properly.
• ML (Machine Learning) models are capable of dealing with non-linear temporal variations in electrical load data, but have drawbacks associated with vanishing gradient, overfitting, and complex hyper-parameters tuning problems.
• Hybrid ML models extract prominent features but have complicated architectures and unknown number of optimal clusters that may restraint validity.
• ELM (Extreme Learning Machine) with LM (Levenberg–Marquardt) and CMIFS (Conditional Mutual Information-based Feature Selection) enhances the performance of FFNN (Feed Forward Neural Network) but increases time complexity.
• PSO (Particle Swarm Optimization) and GA (Genetic Algorithm) based NN (Neural Network) have been presented to optimize weights of step-ahead and multi-step ahead STLF models, but these metaheuristic algorithms can easily converge to a local optimum under diversified feature space.
• DL (Deep Learning) methods have enhanced the accuracy of STLF models using highly diversified input data, but have limitations such as the vanishing gradient issue and inadequacy of capturing long-term dependencies.

Proposed Model

• The proposed model is a hybrid DNN methodology for the optimal solution of the STLF problem.
• The hybrid DL model reduces the prediction error in STLF with the increase of high dimensional features and variational parameters, such as weather conditions, timestep, and historical load values.

Significance

• The proposed hybrid DL model captures all the necessary input parameters required for electrical load forecasting, such as temporal and climatic features, and patterns in historical electrical load data.
• The model is expected to improve the accuracy of STLF predictions and provide electric utilities with a state-of-the-art solution for their operating strategies.

Proposed Model

• The architecture is a combination of two NN architectures, DCCN (Dilated Causal Convolution Neural Network) and BiLSTM (Bidirectional Long Short-Term Memory), with four modifications introduced to address over-fitting and overwhelming of model parameters.
• The encoder section of the model employs a novel SRDCC (Short Receptive field based Dilated Causal Convolutional network) module with small filter sizes for effective capture of both generalized and specific trends in the electrical load data.
• For step-ahead load forecasting, the first one-dimensional convolutional layer uses 1x1 dilated causal convolution filter to maximize the capability of capturing the specific nonlinear local patterns in the electrical load data. The second layer uses 2x2 dilated causal convolution filter to capture generalized trends with small model parameters.
• For multi-step ahead forecasting, 1x1 dilated causal convolution filter is used in both the first and second convolutional layers with dilation rates set to 1 and 2, respectively, to capture all possible local trend extraction from the electrical load data.
• In the decoder section, a BiLSTM module is used to record the long-term dependencies in the data, with 128 neurons and linear activation function to avoid under-fitting and overfitting problems.
• The output layer of the model consists of a single neuron for predicted electrical load value.
• The model is trained, validated, and tested using a dataset normalized using the standard min-max scaling technique, with the Adam optimizer used during the training and validation phase. The proposed model is evaluated for effectiveness and compared with standard LSTM (Long Short-Term Memory) model, CNN (Convolutional Neural Network)-LSTM, and ANN (Artificial Neural Network) for a fair comparison.

Experiment

Quantitative comparison of Linear Parametric Models for Step-Ahead STLF

• ARMAX (AutoRegressive Moving Average model with eXogenous inputs) performs better than OE (Output Error) and ARX (AutoRegressive with eXogenous inputs) in presenting strong autocorrelation of electrical load data and capturing the local trend between input parameters.

Quantitative comparison of machine learning models for step-ahead electrical load forecasting

• The KNN (K-Nearest Neighbors) shows the worst performance among all ML models due to its overfitting problem and lack of regularization effect.
• ANN-LM models with one and two dense hidden layers outperform other ML algorithms, with one dense layer being comparatively simpler and able to extract both linear and non-linear relationships between seasonal trends and electrical load curve convincingly.

Quantitative comparison of hybrid DL models for step-ahead electrical load forecasting

• The CNN-LSTM model has better tendency to capture local trends and non-linear relationships between diversified features and electrical load patterns compared to LSTM model.

Quantitative comparison of alternative models for day-ahead short-term electrical load forecasting

• The presented results show that the proposed SRDCC-BiLSTM model performs comparatively better than other ML and hybrid DL models for multi-step ahead electrical load forecasting, capturing most of the peak and valley load patterns efficiently while producing stable and refined predicted output.