Universality of Gradient Descent Neural Network Training

The paper proves a universality theorem: for any neural network for which some algorithm can produce effective weights on a classification task, one can construct an extended network whose weights and outputs are obtained solely by gradient-descent training. The construction is theoretical and not intended for practical use, but it illustrates the potential reach of meta-learning techniques.