website design software


1. Physically grounded Scene interpretation

object_layoutMost  environments around us have high regularity for instance indoor scenes, where objects don't occur randomly but follow certain spatial rules. Most of the current recognition methods treat objects as mere 2D templates and consequently can not benefit from the constraints offered by the physical world.  We propose a 3D based representation for objects that is consistent with the surrounding scene structure. Such representation facilitates modeling of richer spatial interactions between objects and  the scene  in addition to providing an approximate 3D localisation of an object, that is informative of the usable space in the scene.

[project] [paper]

2. Recovering the Spatial Layout of Cluttered Rooms

We consider the problem of recovering the spatial layout of indoor scenes from monocular images. Clutter is a major problem for existing singleview 3D reconstruction algorithms. We gain robustness to clutter by modeling the global room space with a parameteric 3D “box” and by iteratively localizing clutter and refitting the box.

[project] [paper]

3. Physically grounded photo editing

object_insertion Current image editing software only allows 2D manipulations with no regard to the high level spatial information that is present in a given scene, and 3D modeling tools are sometimes complex and tedious for a novice user. Our goal is to extract 3D scene information from single images to allow for seamless object insertion, removal, and relocation. This process can be broken into three somewhat independent phases: luminaire inference, perspective estimation (depth, occlusion, camera parameters), and texture replacement. We are working on developing novel solutions to each of these phases, in hopes of creating a new class of physically-aware image editors.

[project] [paper]

4. Matching segmentation based image representations

Partial_region_matchingRegions obtained from segmentation methods often exhibit poor repeatability. Merging and splitting of segments capturing same object in the scene under different lighting or viewpoints makes it difficult to find region correspondences using one-to-one matching algorithms. We present a partial region matching approach as a solution to this problem, combined with a many to one label assignment framework to
match regions under adjacency constraints.

[project] [paper]