Jürgen Schmidhuber

Career Schmidhuber taught there from 2004 until 2009. From 2009 to 2021, he was a professor of artificial intelligence at the Università della Svizzera Italiana in Lugano, Switzerland. He has served as the director of Dalle Molle Institute for Artificial Intelligence Research (IDSIA), a Swiss AI lab, since 1995. Since 2021, he has also been the director of the AI Initiative at the King Abdullah University of Science and Technology (KAUST).

In 2014, Schmidhuber formed a company, NNAISENSE, to work on commercial applications of artificial intelligence in fields such as finance, heavy industry and self-driving cars. Sepp Hochreiter, Jaan Tallinn, and Marcus Hutter are advisers to the company. Sales were under US$11 million in 2016; however, Schmidhuber states that the current emphasis is on research and not revenue. NNAISENSE raised its first round of capital funding in January 2017. Schmidhuber's overall goal is to create an all-purpose AI by training a single AI in sequence on a variety of narrow tasks, but as of 2026 he has said that the focus of NNAISENSE has shifted from artificial general intelligence to asset management.

Research

In the 1980s, backpropagation did not work well for deep learning with long credit assignment paths in artificial neural networks. To overcome this problem, Schmidhuber (1991) proposed a hierarchy of recurrent neural networks (RNNs) pre-trained one level at a time by self-supervised learning. It uses predictive coding to learn internal representations at multiple self-organizing time scales, facilitating downstream deep learning. The RNN hierarchy can be collapsed into a single RNN, by distilling a higher level chunker network into a lower level automatizer network. In 1993, a chunker solved a deep learning task whose depth exceeded 1000.

In 1991, Schmidhuber published adversarial neural networks that contest with each other in the form of a zero-sum game, where one network's gain is the other network's loss. which he considered "one of the most important documents in the history of machine learning". The standard LSTM architecture was introduced in 2000 by Felix Gers, Schmidhuber, and Fred Cummins. LSTM using backpropagation through time was published with his student Alex Graves in 2005, and its connectionist temporal classification (CTC) training algorithm in 2006. CTC was applied to end-to-end speech recognition with LSTM.

In 2014, the state of the art was training "very deep neural network" with 20 to 30 layers. Stacking too many layers led to a steep reduction in training accuracy, known as the "degradation" problem. In May 2015, Rupesh Kumar Srivastava, Klaus Greff, and Schmidhuber used LSTM principles to create the highway network, a feedforward neural network with hundreds of layers, much deeper than previous networks. In Dec 2015, the residual neural network (ResNet) was published, which is a variant of the highway network.

In 1992, Schmidhuber published fast weights programmer, an alternative to recurrent neural networks. It has a slow feedforward neural network that learns by gradient descent to control the fast weights of another neural network through outer products of self-generated activation patterns, and the fast weights network itself operates over inputs.

In 2011, Schmidhuber's team at IDSIA with his postdoc Dan Ciresan also achieved dramatic speedups of convolutional neural networks (CNNs) using graphics processing units (GPUs), based on CNN designs introduced much earlier by Kunihiko Fukushima. The deep CNN of Dan Ciresan et al. (2011) at IDSIA was 60 times faster and achieved the first superhuman performance in a computer vision contest in August 2011. Between 15 May 2011 and 10 September 2012, these CNNs won four more image competitions and improved the state of the art on multiple image benchmarks. The approach has become central to the field of computer vision. and published details of numerous priority disputes with Hinton, Bengio and LeCun.

The term "schmidhubered" has been jokingly used in the AI community to describe Schmidhuber's habit of publicly challenging the originality of other researchers' work, a practice seen by some in the AI community as a "rite of passage" for young researchers. Some suggest that Schmidhuber's significant accomplishments have been underappreciated due to his confrontational personality.

Recognition

Schmidhuber received the Helmholtz Award of the International Neural Network Society in 2013, and the Neural Networks Pioneer Award of the IEEE Computational Intelligence Society in 2016 for "pioneering contributions to deep learning and neural networks." the "father of generative AI", and the "father of deep learning". and gives credit to many even earlier AI pioneers.

Views Schmidhuber is a proponent of open source AI, and believes that they will become competitive against commercial closed-source AI.