Debidatta Dwibedi

I am a Research Scientist in Google Brain. I completed my Masters in Robotics from the Robotics Institute at CMU, where I was advised by Martial Hebert. Prior to that I completed my undergrad from IIT Kanpur, where I worked with Amitabha Mukerjee.

Email  /  Google Scholar  /  GitHub /  LinkedIn  /  Twitter

Publications  /  Patents  /  Talks /  Theses /  Misc


I want to build intelligent agents that interact with our world in useful ways.

My research lies at the intersection of machine learning, computer vision and robotics. Presently, I am working on imitation learning from videos and investigating the role time can play in learning better visual models.


Counting Out Time: Class Agnostic Video Repetition Counting in the Wild
Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, Andrew Zisserman
Computer Vision and Pattern Recognition (CVPR) 2020

Count repetitions in videos in a class-agnostic manner.

paper | abstract | bibtex | project | teaser video | Google AI blogpost | colab

Temporal Cycle-Consistency Learning
Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, Andrew Zisserman
Computer Vision and Pattern Recognition (CVPR) 2019

Self-supervised representation learning based on temporal alignment for fine-grained video understanding tasks.

paper | interactive paper | abstract | bibtex | project | poster | Google AI blogpost | code | colab

Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
Ilya Kostrikov, Kumar Krishna Agrawal , Debidatta Dwibedi, Sergey Levine , and Jonathan Tompson
International Conference on Learning Representations (ICLR) 2019

Sample efficient imitation learning using off-policy updates and proper handling of terminal states.

paper | abstract | bibtex | code

Learning Actionable Representations from Visual Observations
Debidatta Dwibedi, Jonathan Tompson, Corey Lynch and Pierre Sermanet
International Conference on Intelligent Robots (IROS) 2018

Control agents from pixels by learning self-supervised representations from videos.

paper | abstract | bibtex | project

Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
Debidatta Dwibedi, Ishan Misra and Martial Hebert
International Conference on Computrer Vision (ICCV) 2017

Generate synthetic data for detecting objects in scenes.

paper | abstract | bibtex | code | poster

Deep Cuboid Detection: Beyond 2D Bounding Boxes
Debidatta Dwibedi, Tomasz Malisiewicz, Vijay Badrinarayanan and Andrew Rabinovich
Arxiv Preprint, 2016

Cuboid detector using deep learning: finds cuboids in scenes and localizes their corners.

paper | abstract | bibtex

Characterizing Predicate Arity and Spatial Structure for Inductive Learning of Game Rules
Debidatta Dwibedi and Amitabha Mukerjee
ECCV 2014 Workshop on Computer Vision + Ontology Applied Cross-Disciplinary Technologies 2014

Represent videos as dynamic graphs. Learn rules of games from observing people play games in Kinect videos.

paper | abstract | bibtex | videos


Deep learning system for cuboid detection


Temporal Cycle-Consistency Learning
Learning from Unlabeled Videos at CVPR 2019

Temporal Reasoning in Videos Using Convolutional Gated Recurrent Units
2nd Workshop in Brave New Ideas in Video Understanding at CVPR 2018

paper | slides | poster | bibtex

Self-Supervised Representation Learning for Continuous Control
3rd Workshop in Machine Learning in the Planning and Control of Robot Motion at ICRA 2018


Synthesizing Scenes for Instance Detection
How can we create annotated datasets without humans for tasks like object detection and pose estimation?

Observational Learning of Rules of Games
Can we learn the rules of a game by observing people playing them?


Some other unpublished work:

Playing Games with Deep Reinforcement Learning

Towards Pose Estimation of 3D Objects in Monocular Images via Keypoint Detection

HandNet: Using Faster R-CNN to Detect Hands in Egocentric Videos

A Grounded Framework for Gestures and its Applications

this guy's webpage is awesome