| Abstract: |
Several studies have discussed the impact different optimization techniques in
the context of time series forecasting across different Neural network
architectures. This paper examines the effectiveness of Adam and Nesterov's
Accelerated Gradient (NAG) optimization techniques on LSTM and GRU neural
networks for time series prediction, specifically stock market time-series.
Our study was done by training LSTM and GRU models with two different
optimization techniques - Adam and Nesterov Accelerated Gradient (NAG),
comparing and evaluating their performance on Apple Inc's closing price data
over the last decade. The GRU model optimized with Adam produced the lowest
RMSE, outperforming the other model-optimizer combinations in both accuracy
and convergence speed. The GRU models with both optimizers outperformed the
LSTM models, whilst the Adam optimizer outperformed the NAG optimizer for both
model architectures. The results suggest that GRU models optimized with Adam
are well-suited for practitioners in time-series prediction, more specifically
stock price time series prediction producing accurate and computationally
efficient models. The code for the experiments in this project can be found at
https://github.com/AhmadMak/Time-Series-Optimization-Research Keywords:
Time-series Forecasting, Neural Network, LSTM, GRU, Adam Optimizer, Nesterov
Accelerated Gradient (NAG) Optimizer |