ENERGY DEMAND FORECASTING USING LSTM WITH DATA PRE-PROCESSING IN APACHE SPARK

There has been a strong growth in the global demand for energy over the past decade, especially in relation to renewable energy. The accelerating use of energy by household appliances, rapidly rising sales of air conditioners especially in emerging economies and the ever increasing need for heating in households has put a strain on energy demands across the world. As such, it has become more pertinent for energy producers to be able to accurately predict demand to avoid either over or under supply of electricity to households. In this paper, Apache Spark is used to process London smart meter data collected over a period of a year. The processed time-series data is then used to train a LSTM model for energy demand forecasting. When compared to using a single machine, the Spark cluster performed the same data processing task at a much shorter time, supporting the idea that its parallel architecture is more suitable for processing large datasets.