Address: Microsoft Research,
300 Lafayette Street,
New York, NY 10012

Dipendra Misra

I am a Senior Researcher at Microsoft Research, New York. I received my PhD in computer science from Cornell University (2019) and my bachelors in computer science from Indian Institute of Technology Kanpur (2013).

Research Interests: My main interest is in developing efficient machine learning algorithms with applications to real-world problems. The word efficient here includes provable, sample and computationally efficient, interpretable, scalable, and ethical. My empirical focus is on problems in natural language understanding and allied fields. I am currently active in reinforcement learning theory, interactive learning, representation learning, and language and vision problems. I also have interest in computational social science, and data and society.

News: We have a new paper on interactive learning at ICML 2021 that only uses language feedback (i.e., no reward, actions, etc.). Our algorithm uses language descriptions of trajectories and solves a sequence of supervised learning problems. We evaluate on grounded language understanding tasks and provide convergence guarantees. [arXiv paper]

News: Qinghua Liu and I recently gave a talk at RL theory seminar. You can find it here.

Provable RL: We have three new provable reinforcement learning algorithms for rich-observation problems. These algorithms are computationally efficient, and their sample complexity is either independent of the size of observation space, or only weakly depends on it.

  • FactoRL at ICLR 2021 that provably solves a subset of rich-observation problems with a latent exponentially large state space. FactoRL learns the latent factorized model, and a state decoding function.
    [Paper] [Code coming soon]
  • RichID at NeurIPS 2020 that provably solves continuous control problem with latent LQR dynamics and rich-observations. RichID learns the latent LQR model and a near-optimal policy.
    [arXiv Version] [Code]
  • Homer at ICML 2020 that solves rich-observation problems with a discrete latent state space. Homer provably explores, recovers the latent dynamics, and optimizes any given reward function.
    [arXiv Version] [ICML Version] [Code] [Medium Post]
We are hiring!
  • For post-doc and full-time positions in reinforcement learning apply here.

Quick Links:   MSR Reinforcement Learning,   Cereb-RL Code Base,   CIFF Code Base,   My Blog,   RL Formulas

Publications



New Preprint

Provable RL with Exogenous Distractors via Multistep Inverse Dynamics
Yonathan Efroni, Dipendra Misra, Akshay Krishnamurthy, Alekh Agarwal, and John Langford
[arXiv 2021] [Code to come soon]

Investigating the Role of Negatives in Contrastive Representation Learning
Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, and Dipendra Misra     (alphabetic ordering)
[arXiv 2021] [Code to come soon]

Have you tried Neural Topic Models? Comparative Analysis of Neural and Non-Neural Topic Models with Application to COVID-19 Twitter Data
Andrew Benett, Dipendra Misra, and Nga Than     (alphabetic ordering)
(Early version accepted at KDD 2021 Workshop)
[arXiv 2021] [Code]



Conference

Interactive Learning from Activity Description
Khanh Nguyen, Dipendra Misra, Robert Schapire, Miro Dudík, Patrick Shafto
In Proceedings of the 38th International Conference on Machine Learning (ICML), 2021.
[Paper] [Version at EML workshop, ICLR 2021] [Code]

Provable Rich Observation Reinforcement Learning with Combinatorial Latent States
Dipendra Misra, Qinghua Liu, Chi Jin, John Langford
In Proceedings of the 9th International Conference on Learning Representations (ICLR), 2021.
[Paper] [Code to Come Soon]

Learning the Linear Quadratic Regulator from Nonlinear Observations
Zakaria Mhammedi, Dylan J. Foster, Max Simchowitz, Dipendra Misra, Wen Sun, Akshay Krishnamurthy, Alexander Rakhlin, John Langford
In Proceedings of the 34th Conference on Neural Information Processing Systems (NeuRIPS), 2020.
[arXiv Version] [NeuRIPS Version] [Code]

Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learning
Dipendra Misra, Mikael Henaff, Akshay Krishnamurthy, and John Langford
In Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.
[arXiv Version] [ICML Version] [Code]

Early Fusion for Goal Directed Robotic Vision
Aaron Walsman, Yonatan Bisk, Saadia Gabriel, Dipendra Misra, Yoav Artzi, Yejin Choi, Dieter Fox
In International Conference on Intelligent Robots and Systems (IROS), 2019.
[Paper]    [Best paper nomination]

Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, Yoav Artzi
In Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[Paper] [Dataset and SDR Code] [Navigation Code]

Mapping Navigation Instructions to Continuous Control Actions with Position Visitation Prediction
Valts Blukis, Dipendra Misra, Ross A. Knepper, and Yoav Artzi
In Proceedings of the Conference on Robot Learning (CoRL), 2018.
[Paper] [Code] [Demo Video]

Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations
Dipendra Misra, Ming-Wei Chang, Xiaodong He and Wen-tau Yih
In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018.
[Paper] [Code]

Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction
Dipendra Misra, Andrew Bennett, Valts Blukis, Eyvind Niklasson, Max Shatkhin, and Yoav Artzi
In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018.
[Paper] [Code, Data and Simulators]

Lipschitz Continuity in Model-based Reinforcement Learning
Kavosh Asadi*, Dipendra Misra*, Michael L. Littman (* equal contribution)
In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018.
[Paper] [Code]

Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
Dipendra Misra, John Langford and Yoav Artzi
In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2017.
[Paper] [Code] [Arxiv Preprint]

Neural Shift-Reduce CCG Semantic Parsing
Dipendra Misra and Yoav Artzi
In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2016.
[Paper] [Supplementary] [Code]

Environment-driven lexicon induction for high-level instructions
Dipendra K. Misra, Kejia Tao, Percy Liang, Ashutosh Saxena
In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2015.
[Paper] [Supplementary] [Code] [Data] [Simulator] [Bibtex]

Robo Brain: Large-Scale Knowledge Engine for Robots
Ashutosh Saxena, Ashesh Jain, Ozan Sener, Aditya Jami, Dipendra K. Misra, Hema S Koppula
In Proceedings of the International Symposium of Robotics Research (ISRR), 2015.
[Paper] [Website]

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions
Dipendra K. Misra, Jaeyong Sung, Kevin K. Lee, Ashutosh Saxena
In Proceedings of the Robotics: Science and systems (RSS), 2015.
[Paper] [Website] [Simulator] [Bibtex]


Old Preprint

Combating the Compounding-Error Problem with a Multi-step Model
Kavosh Asadi, Dipendra Misra, Seungchan Kim, Michel L Littman
arXiv, 2019.
[Paper]


Journal

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions
Dipendra K. Misra, Jaeyong Sung, Kevin K. Lee, Ashutosh Saxena
In The International Journal of Robotics Research (IJRR), 2015.
[Paper] [Website] [Bibtex]




Workshop

Towards a Simple Approach to Multi-step Model-based Reinforcement Learning
Kavosh Asadi, Evan Carter, Dipendra Misra, Michael Littman
Deep Reinforcement Learning Workshop at the Conference on Neural Information Processing Systems (NeurIPS), 2018.
[Paper]

The Third Workshop on Representation Learning for NLP (Rep4NLP)
Isabelle Augenstein, Kris Cao, He He, Felix Hill, Spandana Gella, Jamie Kiros, Hongyuan Mei and Dipendra Misra
Workshop at the Annual Meeting of the Association for Computational Linguistics (ACL), 2018.
[Workshop Proceedings]

Equivalence Between Wasserstein and Value-Aware Model-based Reinforcement Learning
Kavosh Asadi, Evan Carter, Dipendra Misra and Michael L. Littman
Workshop on Prediction and Generative Modeling in Reinforcement Learning (PGMRL) at the International Conference on Machine Learning (ICML), 2018.
[ArXiv Preprint]

CHALET: Cornell House Agent Learning Environment
Claudia Yan, Dipendra Misra, Andrew Bennett, Aaron Walsman, Yonatan Bisk and Yoav Artzi
arXiv report, 2018.
[Paper] [Website] [Bibtex]

Reinforcement Learning for Mapping Instructions to Actions with Reward Learning
Dipendra Misra and Yoav Artzi
Symposium on Natural Communication for Human-Robot Collaboration at AAAI Fall Symposium Series, 2017.
[Paper] [Code]

Posts

  • Coronavirus and Hosting Conferences Remotely     [Post]

  • Academia and Compute-Intensive AI Research    [Post]

  • PAC with Hoeffding-Bernstein    [Post]

  • Growing Bifurcation of AI Scholarship     [Post]

  • Dynkin’s π-λ Theorem and CDF     [Part 1]     [Part 2]

  • Are Synthetic Datasets in AI Useful?     [Post]

  • Are we doing NLP the right way?     [Post]

  • Writing and Proof Reading Research Code     [Post]

  • Writing Strong PhD Applications     [Post]

  • Mathematical Analysis of Policy Gradient Methods     [Post]

  • Tutorial on Markov Decision Process Theory and Reinforcement Learning.     [Slides Part 1]     [Slides Part 2]     [Post]