Staff Research Scientist,
Mosaic Research, Databricks
Dipendra Misra
I am a machine learning researcher specializing in the field of natural language understanding, interactive learning (e.g., reinforcement learning), and representation learning. My main research agenda is to develop generalizable agents that can interact with the world using actions and natural language, and solve a range of tasks using reward, natural language feedback, or other types of feedback.
A recurring theme in my research is developing interactive learning algorithms or representation learning methods using feedback or data that naturally occurs in real-world such as video data, user edits, language feedback, etc. My current areas of research focus are below.
(Algorithms) Learning Algorithms for LLMs and Foundation Model: I am interested in developing both practical and efficient algorithms for training agents. In particular, my recent focus has been on developing algorithms for fine-tuning LLMs using better imitation learning algorithms (arXiv 2023) such as our DR-PO algorithm (arXiv 2024). My other past work includes developing RL methods that are provably sample-efficient and computationally-efficient: the Homer algorithm (ICML 2020), RichID algorithm (NeurIPS 2020), FactoRL Algorithm (ICLR 2021), and PPE algorithm (ICLR 2022 Oral), and AC State (TMLR 2023).
(Feedback) Developing Agents that Learn from Language Feedback: Once an agent foundation model has been deployed, it may need post-training or adaptation to given a setup. I have developed approaches that can finetune models using language feedback which is easy and natural for non-expert humans to provide. E.g.,
Our recent paper (arXiv 2024) studies aligning LLMs using user edit feedback that is naturally generated in writing assistant applications.
It is more expensive to label a trajectory for a given language instruction than to label instruction for a given trajectory. Our ICML 2024 and ICML 2021 papers train agents using these hindsight language instructions.
Humans often use free-form language feedback to guide each other. Our recent LLF-Bench paper (arXiv 2023) introduces a benchmark for evaluating learning in LLM agents using language feedback.
(Models) Foundation Models for Decision Making: I am focusing on developing foundation models that can take actions for different agents, in different domains, and for different tasks. E.g., an agent that can interact in an OS or a bot that can play a game against a human. I am particularly interested in using naturally available data such as videos for building these models. My relevant recent work is our ICLR 2024 (Spotlight) paper on learning right representations from videos. Other relevant work is my series of work published at EMNLP 2017, EMNLP 2018, CoRL 2018, and CVPR 2019, on developing instruction following agents that can solve a variety of tasks specified in natural language, in embodied setting.
Beyond my main agenda, I also have interest in a diverse range of topics including language and vision problems, statistical learning theory, and computational social science.
Bio: I am a Staff Research Scientist at the Mosaic Research team at Databricks. I received my PhD in computer science from Cornell University (2019). Previously, I was a Senior Researcher at Microsoft Research (2019- Aug 2014). I most closely associate with the ML (ICLR, ICML, NeurIPS) and NLP (ACL, EMNL, CoNLL) research communities. I am Senior Area Chair for ACL 2025 and was Area Chair for ICLR 2025 and CoNLL (2021, 2022).
Quick Links: Databricks Mosaic Research, Intrepid Code Base, CIFF Code Base, Math for AI, My Blog, RL Formulas
