MURI reports - 2011

Caltech Vision Lab - 2010-11 MURI projects

Behavior Recognition of Mice

People

X.P. Burgos-Artizzu, P. Dollar, P. Perona

Goal

Create system for automatic annotation of rodent behavior to assist biologists.

Progress 2010-11

Development of a new automatic classification system that is able to recognize instances of up to 16 different behaviors in full unsegmented videos. The system recognizes behaviors using features from both mouse trajectories (computed using the previously developed tracking system) as well as spatio-temporal words. Ran algorithm on hundreds of hours of video, reaching an average 55% recognition rate. Typical human agreement on same videos is of about 60-70%. We will continue improving the system until it can perform as well as human annotators.

Fast Robust Object Detection

People

R. Appel, P. Dollar, P. Perona

Goal

Create system that achieves state-of-the-art detection but with great gain in speed

Progress 2010-11

Comparing various top-performing methods, specifically experimenting with and identifying which aspects of these systems can be sped up using more efficient algorithms. Introduced new Oriented Airplanes Dataset (7 aiports, ~1000 airplanes, ~4000 test images) for oriented detection experiments. Determined that simple feature shapes are just as good as complex shapes but are much faster to compute and are easily steerable. Incorporating these findings into a novel system which implicitly takes care of part deformations (shifts or rotations).

Methods For Electronic Olfaction In The Wild

People

G. Griffin, P. Perona, N. Lewis

Goal

Enable electronic olfaction of large numbers of odors in situ using features optimized to be discriminative in under outdoor conditions.

Progress 2010-11

Over 64/32 odors have been acquired indoors/outdoors. Our current feature set called a "sniffy" modulates incoming odors at a variety of frequencies in order to exploit discriminative time-sensitive properties of the detectors. As with images categories, some odor categories are easily distinguishable while other are confounded. In vision we have found that cross-category confusion can be used to construct hierarchies which, in turn, can be exploited for rapid detection (G. Griffin & P. Perona CVPR 2008). We are currently investigating the efficiency of similar techniques in olfaction.

Modeling the Image Annotation Process

People

P. Welinder, S. Branson, S. Belongie, P. Perona

Goal

Principled models for the image annotation process in order to achieve cost-effective crowdsourcing.

Progress 2010-11

Developed a model for confidence annotations to theoretically analyze the optimal number of labels on a confidence scale. Experiments in merging detection and localization annotations from multiple annotators.

Publications

• Welinder, P., Branson, S., Belongie, S., Perona, P. The Multidimensional Wisdom of Crowds. NIPS. 2010
• Welinder, P., Perona, P. Online crowdsourcing: rating annotators and obtaining cost-effective labels. Workshop on Advancing Computer Vision with Humans in the Loop at CVPR. 2010.

• Project Page: http://www.vision.caltech.edu/welinder/cubam.html

Fear in Flies

People

P. Felsen, C. Fernandez, P. Perona, D. Anderson

Goal

Create a system for automatic fly detecting/tracking and analysis to assist biologists.

Progress 2010-11

Continued development of a robust fly detector/tracker for low resolution multi-fly videos. Additions include: identify individual flies in fly clusters, flag frames with possible fly detection errors, allow the user to manually modify fly detections and tracks, compute statistics for a number of fly behaviors, etc. We made fly behavior observations that will be further explored in new, specifically designed experiments.

ABC for CV

People

Th.J.Fuchs, L. Matthies, P. Perona

Goal

3D object recognition and pose estimation with uncertainty quantification.

Progress 2011

Approximate Bayesian Computation is employed to infer the parameters of 3D objects in a scene utilizing multimodal input, ranging from RGB images to LIDAR measurements. The posteriors over all parameters can be used for a MAP pose estimation and for uncertainty quantification and propagation, e.g. for robotic manipulation.

Funding

SNSF

Probabilistic Terrain Modeling

People

Th.J.Fuchs, P. Perona, L. Matthies

Goal

Probabilistic terrain modeling for rover path planing and Martian landing site selection

Progress 2011

The main goal of the project is to develop a framework for learning probabilistic terrain models from orbital data, which can be updated on-line by ground data from traversing rovers. The prime application areas are landing site selection for future rover missions and large scale path planing by predicting sink and slip from the terrain type.

Funding

SNSF

Fly aggression detection

People

A. Lashgari, P. Perona, M. Maire

Goal

Detection of fly aggression

Progress

Fly aggression is one of the more challenging behaviors of flies to detect compared to other behaviors like walking or chasing. We have successfully modeled male and female aggression using probabilistic and graphical models. The recognition rate is comparable to human detection ability and we hope to test the method for other experimental settings of flies or other animals.

Object Detection and Segmentation

People

M. Maire, P. Perona

Goal

Utilize low-level image segmentation to enhance object detection.

Progress 2011

We developed a new framework in which image segmentation, figure/ground organization, and object detection all appear as the result of solving a single grouping problem. This framework serves as a perceptual organization stage that integrates information from low-level image cues with that of high-level part detectors. Pixels and parts each appear as nodes in a graph whose edges encode both affinity and ordering relationships. We derive a generalized eigenproblem from this graph and read off an interpretation of the image from the solution eigenvectors. Combining an off-the-shelf top-down part-based person detector with our low-level cues and grouping formulation, we demonstrate improvements to object detection and segmentation.

Publications

Maire, M., Yu, S.X., Perona, P. Object Detection and Segmentation from Joint Embedding of Parts and Pixels. ICCV 2011.