Recognizing Objects and their Properties

We want visual systems that can provide detailed descriptions and robust predictions of objects. In some cases, it may be sufficient to name the object. In others, we would like the computer to localize parts and infer pose. In many others, the object may be unfamiliar, and a rough description should suffice. For example, if an assisted driving program encounters a cow on the road, it should recognize it as a four-legged animal and predict its movement, even if it has not seen a cow before.

This project includes the following goals:

• Build recognition systems that can recognize everything. Rather than applying detectors for specific categories of interest, the system should be able to localize objects, describe them, and infer coarse categories. With more experience, the system should get more detailed.
• Precisely localize objects and infer their shape properties and functional affordance. Segmentation is important for inferring shape from contour, and shape is important to know what an object could be used for.
• Create systems that are easily extendible and that learn more quickly with experience. We want systems that incrementally learn new categories, parts, and materials, building on what it already knows. This requires creating flexible learning techniques and good ways to relate different recognition tasks.

Resources

BCP Part-based Detection Code: from CVPR'13 paper

Detection Analysis Code: from ECCV'12 paper

CORE Dataset: includes segmentations of objects, parts, and materials for thousands of images

Attributes Dataset: includes binary attribute labels for PASCAL 2007 images

Fast SVM Training Code: good online learning algorithm for histogram intersection kernels

Publications and Projects

Learning Collections of Part Models for Object Recognition
Ian Endres, Kevin Shih, Johnston Jiaa, and Derek Hoiem
CVPR, 2013. [pdf] [project]

Boundary Cues for 3D Object Shape Recovery
Kevin Karsch, Zicheng Liao, Jason Rock, Jonathan T. Barron, and Derek Hoiem
CVPR, 2013. [pdf] [supp]

Improved Object Categorization and Detection Using Comparative Object Similarity
Gang Wang, David Forsyth, and Derek Hoiem
PAMI Vol. 99, 2013. [pdf]

Diagnosing Error in Object Detectors
Derek Hoiem, Yodsawalai Chodpathumwan, and Qieyun Dai
ECCV, 2012. [pdf] [supp (40MB)] [src/data (70MB)] [slides]

Learning Shared Body Plans
Ian Endres, Vivek Srikumar, Ming-wei Chang, and Derek Hoiem
CVPR, 2012. [pdf]

Learning to Localize Detected Objects
Qieyun Dai and Derek Hoiem
CVPR, 2012. [pdf]

Attribute-Centric Recognition for Cross-Category Generalization
A. Farhadi, I. Endres, and D. Hoiem
CVPR 2010. [pdf] [CORE dataset]

Describing Objects by their Attributes
A. Farhadi, I. Endres, D. Hoiem, and D.A. Forsyth
CVPR 2009. [pdf] [project]

The Benefits and Challenges of Collecting Richer Object Annotations
Ian Endres, Ali Farhadi, Derek Hoiem, and David Forsyth
ACVHL 2010 (in conjunction with CVPR). [pdf] [CORE dataset]

Comparative object similarity for improved recognition with few or no examples
G. Wang, D.A. Forsyth, D. Hoiem
CVPR 2010. [pdf]

Learning Image Similarity from Flickr Groups Using Stochastic Intersection Kernel Machines
G. Wang, D. Hoiem, and D.A. Forsyth
ICCV 2009. [pdf] [project]

Building Text Features for Object Image Classification
G. Wang, D. Hoiem, and D.A. Forsyth
CVPR 2009. [pdf]

3D LayoutCRF for Multi-View Object Class Recognition and Segmentation
D. Hoiem, C. Rother, and J. Winn
CVPR 2007. [pdf]

Funding Sources

NSF Award 10-53768 for "CAREER: Large-Scale Recognition using Shared Structures, Flexible Learning, and Efficient Search" (2011-2015)

ONR MURI Award for "Rich Representations with Exposed Semantics for Deep Visual Reasonings" (2010-2014)

Google Research Award for "Describing Objects by their Attributes from Images" (2009-2010)

Gift from Microsoft

Gift from Intel

Home