Krishna Murthy Jatavallabhula

Structured World Models for Robots

I am a postdoc at MIT CSAIL with Antonio Torralba and Josh Tenenbaum. I received my PhD at Mila, advised by Liam Paull.

My research focuses on designing structured world models for robots: rich, multisensory models of the physical world that enable robots and embodied AI systems to perceive, reason, and act just as humans are able. My work draws upon ideas from robotics, computer vision, graphics, and computational cognitive science; intertwining our understanding of the world with probabilistic inference and deep learning.

My work has been recognized with PhD fellowship awards from NVIDIA and Google, and a best-paper award from IEEE RAL.

I am looking for academic / industry roles this cycle (Jan-May 2024).

I commit 2 hours a week to mentor underrepresented groups in robotics, computer vision, and machine learning. If this is you, and you'd like to discuss research, grad school apps/advice, or anything else, please fill out this form.
I am committed to making this website as accessible as possible to everyone visiting it. If you would like to suggest/request any accessibility-related modifications, I would really appreciate a quick note.

News

Mar 1, 2024 Speaking at the UMD/Microsoft future leaders in robotics and AI seminar series
Jan 29, 2024 6 papers accepted to ICRA 2024.
Jan 23, 2024 Serving as associate editor for IROS and RA-L.
Sep 28, 2023 Another webpage update, featuring new work, including ConceptGraphs.
Feb 12, 2023 Long overdue webpage update, including the featured Conceptfusion work.
Mar 15, 2022 I moved to MIT to start my potsdoc with Josh Tenenbaum and Antonio Torralba.
Mar 11, 2022 Got my PhD with grade: exceptional!
Feb 2, 2022 Serving as associate editor for IROS 2022
Sep 15, 2021 Organizing workshops Diff3D ICCV 2021, and the PRIBR at Neurips 2021.
Sep 5, 2021 Teaching the realistic/advanced image synthesis class at McGill university (Fall 2021).
Jan 22, 2021 Awarded a Google PhD fellowship (declined)
Dec 21, 2020 Organizing the rethinking ML papers workshop at ICLR 2021
Dec 1, 2020 Honored to have received an NVIDIA graduate fellowship for 2021-22
Nov 10, 2020 gradSLAM is available as an open-source PyTorch framework here
Sep 3, 2020 Organizing the robot learning seminar series at Mila
Sep 1, 2020 Organizing the differentiable vision, graphics, physics workshop at Neurips 2020
Jul 6, 2020 Selected to the RSS pioneers cohort for 2020
Jun 5, 2020 Our paper, MapLite, named best paper, IEEE RAL 2019.
Feb 12, 2020 Our paper on fully differentiable dense SLAM will be (virtually) presented at ICRA 2020
Nov 14, 2019 Released NVIDIA Kaolin: a 3D deep learning library

Featured publications

  1. A gif depicting the 3D mapping process implemented as part of ConceptGraphs.

    ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning

    ICRA 2024
  2. A gif depicting the types of multimodal queries (text, image, audio, click) supported by ConceptFusion.

    ConceptFusion: Open-set Multimodal 3D Mapping

    RSS 2023
  3. A few physical systems implemented in gradSim.

    gradSim: Differentiable simulation for system identification and visuomotor control

    ICLR 2021
  4. An overview of the various components of gradSLAM

    gradSLAM: Dense SLAM meets automatic differentiation

    Krishna Murthy JatavallabhulaGanesh Iyer, and Liam Paull
    ICRA 2020