ENHANCING OPTIMIZATION STABILITY IN DEEP MACHINE LEARNING-BASED INTELLIGENT TIME SERIES FORECASTING: A SYSTEMATIC REVIEW OF GRADIENT CLIPPING TECHNIQUES

Abstract

Intelligent time series forecasting has become a vital tool across sectors such as finance, energy, and healthcare, where accurate predictions can enhance operational efficiency and strategic decisions. However, modern deep learning models for time series often encounter optimisation instability, including exploding and vanishing gradients, which can considerably impair forecasting performance. Gradient clipping has become a common technique to address these issues by limiting gradient magnitudes during training. Despite its significance, existing research on gradient clipping is scattered, with different methods, thresholds, and evaluation strategies applied across various architectures and fields. There is currently a lack of a comprehensive synthesis that systematically explores how gradient clipping affects optimisation stability, the effectiveness of different clipping strategies, and their practical implications for intelligent time series forecasting. To address this, this study presents a systematic literature review (SLR) of gradient clipping techniques, aiming to classify clipping strategies, assess their benefits and drawbacks, and identify open research challenges. The results offer a structured taxonomy of clipping methods, highlight areas needing methodological improvement, and suggest directions for future research to improve model stability and forecasting accuracy.