Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks

作者：Sahel Vahedi Noori, Bin Hu, Geir E. Dullerud, Peter J. Seiler · 年份：2024 · DOI：10.1109/cdc56724.2024.10886894 · 被引用次数：5 · 研究领域：Neural Networks and Applications、Machine Learning and ELM、Advanced Algorithms and Applications

This paper presents sufficient conditions for the stability and $\ell_{2}$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated ReLUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a “lifted” representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.