William Fedus

William “Liam” Fedus is a machine learning researcher and entrepreneur. He worked at Google Brain and OpenAI, contributing to research on large language models. At OpenAI, he later became vice president of post-training research. He was a member of the research group that developed ChatGPT.

Fedus subsequently earned a master's degree in physics from the University of California, San Diego, where he collaborated with scientists working at the Large Hadron Collider. He later pursued doctoral studies in computer science at the Université de Montréal, where his research focused on machine learning.

Career

Google Brain

At Google Brain, Fedus conducted research on sparse neural networks and mixture-of-experts architectures, reinforcement learning, and the scaling of language models.

OpenAI

Fedus joined OpenAI in 2022. He was part of the research group that developed ChatGPT and later worked on the company's reasoning models. OpenAI's GPT-4 technical report listed him among the project's contributors. The report identifies Fedus as the data flywheel lead for the reinforcement-learning and alignment work and also lists him among contributors to flagship training runs and ChatGPT evaluations. By 2024, he was vice president of OpenAI's post-training research organization, which worked on methods for refining pretrained models' behavior and performance.

Periodic Labs

In 2025, Fedus left OpenAI and co-founded Periodic Labs with Ekin Doğuş Çubuk, a former Google Brain and Google DeepMind researcher. The company was publicly announced in September 2025 and said that it had raised $300 million in seed financing in a round led by Andreessen Horowitz.

Research

Fedus's research has included methods for increasing neural-network capacity and efficiency, particularly through sparse activation and mixture-of-experts architectures. In these systems, only selected parts of a model are activated for each input, allowing parameter count to increase without a proportional increase in computation.

Fedus, Barret Zoph, and Noam Shazeer introduced the Switch Transformer, a sparsely activated transformer architecture with a simplified routing mechanism. The researchers reported faster pretraining than comparable dense models using similar computational resources and trained models with up to 1.6 trillion parameters. He also co-authored ST-MoE, a study of methods for improving the training stability and transfer performance of sparse mixture-of-experts models.

Fedus later co-authored a review of sparse expert models covering their history, routing methods, training challenges, and applications in large-scale deep learning. The review discussed the use of mixture-of-experts architectures in language models and other machine learning systems.

Fedus also conducted research on reinforcement learning. He was the lead author of a study on experience replay that examined how replay capacity, the age of stored experiences, and the frequency of learning updates affected deep reinforcement-learning systems.

He co-authored a study examining whether modifications to transformer architectures produced consistent improvements across implementations and applications. The authors found that many modifications did not produce consistent improvements when compared under a shared experimental framework.

He was also a co-author of a study on emergent abilities in large language models, which examined capabilities that appeared in larger models but were not observed in smaller models.

Fedus was also among the authors of BIG-bench, a collaborative benchmark comprising more than 200 tasks used to evaluate language models at different scales. The project reported that model performance and calibration generally improved with scale, while some forms of social bias increased in ambiguous settings.

Selected publications

Srivastava, Aarohi; et al. (2023). "Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models". Transactions on Machine Learning Research. arXiv:2206.04615. Fedus, William; Dean, Jeff; Zoph, Barret (2022). “A Review of Sparse Expert Models in Deep Learning.” arXiv:2209.01667. Du, Nan; Huang, Yanping; Dai, Andrew M.; et al. (2022). "GLaM: Efficient Scaling of Language Models with Mixture-of-Experts". Proceedings of the 39th International Conference on Machine Learning. 162: 5547–5569. Wei, Jason; Tay, Yi; Bommasani, Rishi; et al. (2022). "Emergent Abilities of Large Language Models". Transactions on Machine Learning Research. arXiv:2206.07682. Fedus, William; Zoph, Barret; Shazeer, Noam (2022). "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity". Journal of Machine Learning Research. 23 (120): 1–39. arXiv:2101.03961. Zoph, Barret; Bello, Irwan; Kumar, Sameer; et al. (2022). "ST-MoE: Designing Stable and Transferable Sparse Expert Models". arXiv:2202.08906. Narang, Sharan; Chung, Hyung Won; Tay, Yi; Fedus, William; et al. (2021). "Do Transformer Modifications Transfer Across Implementations and Applications?" Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. pp. 5758–5773. doi:10.18653/v1/2021.emnlp-main.465. arXiv:2102.11972. Fedus, William; Ramachandran, Prajit; Agarwal, Rishabh; Bengio, Yoshua; Larochelle, Hugo; Rowland, Mark; Dabney, Will (2020). “Revisiting Fundamentals of Experience Replay.” Proceedings of the 37th International Conference on Machine Learning. 119: 3061–3071. arXiv:2007.06700.