Visak Kumar

Senior Research Scientist, Woven by Toyota

I am a Senior Research Scientist at Woven by Toyota, focusing on the application of machine learning in autonomous driving. Before, I worked at the Toyota Research Institute. I obtained my PhD in robotics from Georgia Tech, where I was advised by Dr. Karen Liu and Dr. Sehoon Ha. My research interests include physics-based computer animation, machine learning and robotics.

Google Scholar LinkedIn CV vishak.vk@gmail.com

Projects

New · 2026

Plans from Demonstrations, Skills from Reinforcement Learning

Our entry to the TOWER challenge (G2A workshop, CoRL 2026): two simulated robot arms transfer, stack, dismantle and fill towers of blocks, seeing the scene only through four cameras. Task plans come from the organizers' demonstrations and contact skills from reinforcement learning. An agentic controller strings the skills together, checking each outcome, and a camera-only student is distilled from this teacher.

Blog post Code

2026

Goal Revisions and Support Failures in Contact-Rich Whole-Body Skill Composition

A simulated humanoid learns arm balances, inversions and single-leg balances from human motion capture. Switching to compatible goal revisions while distilling a whole-body controller substantially improves contact-rich skill composition, but late revisions and unloaded requested supports remain open problems.

Blog post

Sim-to-Real Transfer for Robots

Developed reinforcement learning based algorithms that enable learning bipedal and quadrupedal walking controllers.

Bipedal sim-to-real Quadruped sim-to-real

Learning Safe-Falling Strategies for Humanoids

We develop an off-policy reinforcement learning algorithm with a mixture of actor-critic experts to teach robots how to fall by minimizing impulse.

Paper

Improving Humanoid Safety by Combining Model-Based Control with Reinforcement Learning

We develop an algorithm to teach robots to balance without falling when an external push is applied. We combine model-based control inputs with model-free policy learning to improve performance, and present a curriculum that enables efficient learning.

Paper

Joint Space Control

We develop a curriculum learning algorithm to learn joint space control of robot manipulators that achieves low end-effector error (<1 cm), generates smooth control comparable to operational space control, and avoids obstacles.

Paper

Learning to Prevent Falls with an Assistive Device

We develop a reinforcement learning algorithm to learn a fall-prevention control policy for an assistive device such as a lower-limb exoskeleton.

Paper

Visuo-Tactile Control Policy for a Multi-Fingered Robot Hand

We investigated the effect of incorporating different sensing modalities, such as touch and vision, on policy performance in tasks such as grasping on a real-world Kuka-Allegro robot system.

Paper

Learning to Walk on a Treadmill

We learn a control policy to create human agents that can walk on a treadmill in simulation. The biomechanical gait characteristics of the human agent are similar to real-world human walking.

Publications

2023

Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables

M. Xu, P. Huang, Y. Niu, V. Kumar, J. Qiu, C. Fang, K.-H. Lee, X. Qi, H. Lam, B. Li, D. Zhao

International Conference on Artificial Intelligence and Statistics (AISTATS), 2023

PMLR arXiv
2021

Error-Aware Policy Learning: Zero-Shot Generalization in Partially Observable Dynamic Environments

V. Kumar, S. Ha, C. K. Liu

Robotics: Science and Systems (RSS), 2021

arXiv
2021

Joint Space Control via Deep Reinforcement Learning

V. Kumar, D. Hoeller, B. Sundaralingam, J. Tremblay, S. Birchfield

IROS 2021

arXiv
2020

Contextual Reinforcement Learning of Visuo-Tactile Multi-Fingered Grasping Policies

V. Kumar, T. Hermans, D. Fox, S. Birchfield, J. Tremblay

NeurIPS 2020 Workshop

arXiv
2020

Learning a Control Policy for Fall Prevention on an Assistive Walking Device

V. Kumar, S. Ha, G. Sawicki, C. K. Liu

ICRA 2020

arXiv
2019

Sim-to-Real Transfer for Biped Locomotion

W. Yu, V. Kumar, G. Turk, C. K. Liu

IROS 2019

arXiv
2018

Expanding Motor Skills using Relay Networks

V. Kumar, S. Ha, C. K. Liu

Conference on Robot Learning (CoRL), 2018

PMLR
2018

Improving Model-Based Balance Controllers Using Reinforcement Learning and Adaptive Sampling

V. Kumar, S. Ha, K. Yamane

ICRA 2018

IEEE
2017

Learning a Unified Control Policy for Safe Falling

V. Kumar, S. Ha, C. K. Liu

IROS 2017

arXiv

Manuscripts

High Performance Pneumatics using Model Predictive Control

Vikash Kumar, Visak C. V. Kumar, Emanuel Todorov

PDF
Precise Calibration of Robots with Small Link Lengths using Kinematic Extensions

Visak C. V. Kumar, Vikash Kumar

PDF

Experience

May 2022 – Present
Senior Research Scientist · ML Behavior team, Woven by Toyota
Palo Alto, CA
- Designed, trained, and deployed end-to-end machine learning models for autonomous driving.
- Developed machine learning models for vehicle motion prediction.
- Designed a large-scale distributed evaluation framework for end-to-end driving policies, enabling rapid, data-driven iteration and model selection.
- Led the design and implementation of large-scale distributed data curation workflows for ML prediction models, improving operational efficiency and downstream model performance.
- Developed and deployed post-training quantization for efficient on-vehicle inference of large neural policies on edge compute.
Oct 2021 – May 2022
Research Scientist · Toyota Research Institute
Los Altos, CA

Developed a reinforcement learning algorithm for learning robust driving policies in multi-task, multi-agent settings under uncertainty over agent behaviors (AISTATS 2023).
Aug 2016 – Sep 2021
Graduate Research Assistant · Interactive Computing, Georgia Institute of Technology
Atlanta, GA

PhD research on deep reinforcement learning for robot control: error-aware policy learning (RSS 2021), sim-to-real transfer for biped locomotion (IROS 2019), fall prevention with an assistive exoskeleton (ICRA 2020), relay networks (CoRL 2018), and safe falling (IROS 2017).
Summers 2019 & 2020
Research Intern · NVIDIA Research
Redmond, WA

Contextual reinforcement learning of visuo-tactile multi-fingered grasping (NeurIPS 2020 workshop), and curriculum-learned joint-space control with sub-1 cm end-effector error on a robot arm (IROS 2021).
May – Aug 2017
Lab Associate · Disney Research
Pittsburgh, PA

Developed a curriculum learning algorithm to improve a model-based humanoid balance controller using deep reinforcement learning and adaptive sampling.
Dec 2015 – Jun 2016
Research Assistant · CSE Department, University of Washington
Seattle, WA

System identification and model predictive control of pneumatic cylinder actuators for the Adroit hand.

Education

2016 – 2021
Ph.D. in Robotics · Georgia Institute of Technology
Atlanta, GA · advised by C. Karen Liu and Sehoon Ha
2014 – 2016
M.S. in Mechanical Engineering · University of Washington
Seattle, WA · advised by Vikash Kumar and Emo Todorov

Skills

Deep learningPyTorch, TensorFlow

Physics simulatorsMuJoCo, DART / PyDART, Bullet / PyBullet, PhysX, Flex

LanguagesPython, C++

Contact

Email: vishak.vk@gmail.com

LinkedIn Google Scholar