1. Physically grounded Scene interpretation
 Most
environments around us have high regularity for instance indoor
scenes, where objects don't occur randomly but follow certain spatial
rules. Most of the current recognition methods treat objects as mere 2D
templates and consequently can not benefit from the constraints offered
by the physical world. We propose a 3D based representation for
objects that is consistent with the surrounding scene structure. Such
representation facilitates modeling of richer spatial interactions
between objects and the scene in addition to providing an
approximate 3D localisation of an object, that is informative of the
usable space in the scene.
- [project] [paper]
 We
consider the problem of recovering the spatial layout of indoor scenes
from monocular images. Clutter is a major problem for existing
singleview 3D reconstruction algorithms. We gain robustness to clutter
by modeling the global room space with a parameteric 3D “box” and by
iteratively localizing clutter and refitting the box.
[project] [paper]
3. Physically grounded photo editing
Current image editing software only allows 2D manipulations with no
regard to the high level spatial information that is present in a given
scene, and 3D modeling tools are sometimes complex and tedious for a
novice user. Our goal is to extract 3D scene information from single
images to allow for seamless object insertion, removal, and relocation.
This process can be broken into three somewhat independent phases:
luminaire inference, perspective estimation (depth, occlusion, camera
parameters), and texture replacement. We are working on developing novel
solutions to each of these phases, in hopes of creating a new class of
physically-aware image editors.
[project] [paper]
4. Matching segmentation based image representations
 Regions
obtained from segmentation methods often exhibit poor repeatability.
Merging and splitting of segments capturing same object in the scene
under different lighting or viewpoints makes it difficult to find
region correspondences using one-to-one matching algorithms. We present
a partial region matching approach as a solution to this problem,
combined with a many to one label assignment framework to
match regions under adjacency constraints.
[project] [paper]
|
