Senior Research Scientist, Woven by Toyota
I am a Senior Research Scientist at Woven by Toyota, focusing on the application of machine learning in autonomous driving. Before, I worked at the Toyota Research Institute. I obtained my PhD in robotics from Georgia Tech, where I was advised by Dr. Karen Liu and Dr. Sehoon Ha. My research interests include physics-based computer animation, machine learning and robotics.
A simulated humanoid learns arm balances, inversions and single-leg balances from human motion capture. Switching to compatible goal revisions while distilling a whole-body controller substantially improves contact-rich skill composition, but late revisions and unloaded requested supports remain open problems.
Developed reinforcement learning based algorithms that enable learning bipedal and quadrupedal walking controllers.
We develop an off-policy reinforcement learning algorithm with a mixture of actor-critic experts to teach robots how to fall by minimizing impulse.
We develop an algorithm to teach robots to balance without falling when an external push is applied. We combine model-based control inputs with model-free policy learning to improve performance, and present a curriculum that enables efficient learning.
We develop a curriculum learning algorithm to learn joint space control of robot manipulators that achieves low end-effector error (<1 cm), generates smooth control comparable to operational space control, and avoids obstacles.
We develop a reinforcement learning algorithm to learn a fall-prevention control policy for an assistive device such as a lower-limb exoskeleton.
We investigated the effect of incorporating different sensing modalities, such as touch and vision, on policy performance in tasks such as grasping on a real-world Kuka-Allegro robot system.
We learn a control policy to create human agents that can walk on a treadmill in simulation. The biomechanical gait characteristics of the human agent are similar to real-world human walking.
Developed a reinforcement learning algorithm for learning robust driving policies in multi-task, multi-agent settings under uncertainty over agent behaviors (AISTATS 2023).
PhD research on deep reinforcement learning for robot control: error-aware policy learning (RSS 2021), sim-to-real transfer for biped locomotion (IROS 2019), fall prevention with an assistive exoskeleton (ICRA 2020), relay networks (CoRL 2018), and safe falling (IROS 2017).
Contextual reinforcement learning of visuo-tactile multi-fingered grasping (NeurIPS 2020 workshop), and curriculum-learned joint-space control with sub-1 cm end-effector error on a robot arm (IROS 2021).
Developed a curriculum learning algorithm to improve a model-based humanoid balance controller using deep reinforcement learning and adaptive sampling.
System identification and model predictive control of pneumatic cylinder actuators for the Adroit hand.
Email: vishak.vk@gmail.com