Illuminating the black box of reservoir computing

Abstract

Abstract Reservoir computers, using recurrent neural networks with fixed random connections, are known to perform a wide range of information-processing tasks. Yet the transformations taking place within the reservoir, the interaction between input matrix, reservoir, and readout layer, and the influence of key design parameters remain insufficiently understood. Here, we shift the focus from performance maximization to the identification of minimal computational requirements for different model tasks. We investigate how many neurons and how much nonlinearity are needed to solve specific tasks, including cases with non-sigmoidal activation functions. Our results show that the division of labor between input matrix, reservoir, and readout layer depends strongly on the task. In many cases, a weakly coupled and only minimally nonlinear reservoir proves sufficient. In addition, features often considered secondary, such as the structure of the input matrix or the steepness of the activation functions, can become decisive for performance.