Vladlen Koltun

Vladlen Koltun (; born 1980) is an Israeli-American computer scientist and intelligent systems researcher. He currently serves as distinguished scientist at Apple Inc. His main areas of research are artificial intelligence, computer vision, machine learning, and pattern recognition. He also made a significant contribution to robotics and autonomous driving.

Koltun's contributions to research and publications are in the areas of convolutional neural networks, reality simulation, view synthesis, photorealistic rendering, urban self-driving cars simulation, 3D computer graphics, robot locomotion,

He is also known for his work on the photorealism enhancement system, and for the critical study of the Hirsch index, a metric of scientists' work that is common in the community.

Early life and education Koltun was born in 1980 in Kyiv, Ukraine (then part of the Soviet Union), and subsequently grew up in Israel.

In 2000, Koltun graduated magna cum laude from Tel Aviv University with a degree in computer science. He continued his studies at the university and received his PhD with honors in computer science in 2002. His doctoral thesis, Arrangements in four dimensions and related structures, was supervised by Micha Sharir. From 2002 to 2005, he was a postdoctoral fellow at the University of California, Berkeley, where he conducted research in theoretical computer science under the supervision of Christos Papadimitriou.

Career Koltun served as an assistant professor at Stanford University from 2005 to 2013, where he lectured in the areas of computer science, computer graphics, and geometric algorithms. Chaudhuri's work along with Koltun, Evangelos Kalogerakis, and Leonidas Guibas resulted in a SIGGRAPH publication in 2011. As a result, Mixamo licensed the technology from Stanford and later Adobe Inc. acquired Mixamo and further developed Adobe Fuse CC, 3D computer graphics software that enabled users to create 3D characters. In 2014, Koltun joined Adobe to conduct research in visual computing with the primary focus on three-dimensional reconstruction.

Koltun left Adobe to join Intel, where he served in various positions until 2021 for the company's R&D projects for Intelligent Systems.

Research At Intel, Koltun contributed to the development of virtual reality simulators for urban autonomous driving, robots, and drones, focusing on deep reinforcement learning techniques with neural networks in virtual environments. These networks underwent trial-and-error learning in VR before being transferred to robots or drones for real-world applications. This method was applied to the ANYmal robot, a quadrupedal machine with proprioceptive feedback in locomotion control.

The studies in the domain of urban autonomous driving led Koltun's group to the development of the Car Learning to Act (CARLA) project in 2017. It is an open-source simulator, powered by Unreal Engine, that can be used to test self-driving technologies in realistic environments with random dangerous situations. The project was funded by the Intel Labs and Toyota Research Institute.

In 2020, inspired by Google Cardboard, Koltun developed OpenBot along with a German scientist Matthias Müller. It is a software stack that transforms Android smartphones into four-wheeled robots capable of navigation, object tracking, and obstacle avoidance. The robot features a 3D-printable chassis, accommodating a controller, LEDs, a smartphone mount, and a USB cable.

Koltun also contributed to further development in the fields of 3D photorealistic view synthesis and rendering. In 2021, using his work with other researchers at Intel, Enhancing Photorealism Enhancement, a photorealism enhancement system was tested in the Grand Theft Auto 5.

Koltun co-authored a research that developed Swift, an autonomous drone system using onboard sensors that can match the performance of human world champions. The system integrates deep reinforcement learning with real-world data, enabling the drone to perform effectively in physical environments.

Selected works Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, Davide Scaramuzza, Champion-level drone racing using deep reinforcement learning, Nature, vol. 620, August 2023 Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, Dhruv Batra; Habitat: A Platform for Embodied AI Research, International Conference on Computer Vision (2019) Chen Chen, Qifeng Chen, Jia Xu, Vladlen Koltun, Learning to See in the Dark, Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, June 2018

Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio López, Vladlen Koltun; CARLA: An Open Urban Driving Simulator, Conference on Robot Learning (CoRL) 2017 F Yu, V Koltun; Multi-Scale Context Aggregation by Dilated Convolutions, International Conference on Learning Representations (ICLR) 2016 Stephan R Richter, Vibhav Vineet, Stefan Roth, Vladlen Koltun; Playing for Data: Ground Truth from Computer Games, European Conference on Computer Vision (ECCV) 2016 Sergey Levine, Vladlen Koltun; Guided Policy Search, International Conference on Machine Learning (ICML) 2013 Philipp Krähenbühl, Vladlen Koltun; Efficient inference in fully connected CRFs with Gaussian edge potentials, Advances in Neural Information Processing Systems (NIPS) 2011