xLSTM: How Exponential Gating and Matrix Memory Scale Recurrent Neural Networks
xLSTM: How Exponential Gating and Matrix Memory Scale Recurrent Neural Networks For over two decades following its introduction by Hochreiter and Schmidhuber (1997), the Long Short-Term Memory (LSTM) network served as the dominant architecture for sequence modeling. By introducing the constant error carousel and multiplicative gating, LSTMs mitigated the vanishing gradient problem that plagued vanilla recurrent neural networks. However, the emergence of the Transformer architecture (Vaswani et
1 min