Yi's wiki

Computer Vision Based Object Tracking and Detection

Goal

Detect and classify 10-100 categories in urban environments. Emphasys on real time operation.

Techniques:

• Object recognition / classification
• Tracking

Categories: Pedestrian, automobile, bus, truck, street sign, pedestrian crossing, trees

Other categories: luggage, bicycle, motorcycle,

Proposal

Proposal.pdf

Milestones

12/02/7

• Labeled 11 videos, around 19800 frames in total. Note that there are not many trucks and buses shown in the sequences. Also, the car often follows only one or two cars in the 1-minute sequences, so not many independent examples of cars could be collected.

11/24/07

• Completed C Code for the Kernel-based tracker (adaptive scale version). It's accurate and fast, running at 120FPS on the previous testing data (112007). I first ran profile on the Matlab code and wrote 2 C functions to reduce the bottlenecks, then wrote 2 C more functions to further reduce the bottlenecks, and finally I decided to use 2 C functions to cover most computation in C. It took me much longer (3 nights, around 30 hours) than I thought to debug and optimize the C code. The result is identical to TrackingResults112007.avi, but much faster.

11/20/07

• Completed Matlab code for the kernel-based tracker (adaptive scale version). It's quite accurate, but runs slower. I'm still debugging the C code (it's very hard to debug because Matlab simply crashes when there's bug), and hopefully it will run much faster after that.
• Results: TrackingResults112007.avi

11/06/07

• Completed Matlab code for the kernel-based tracker (fixed scale version). It is very accurate and runs at an average of 45FPS.
• Results: TrackingResults110607.avi

12/04/07

• Collect 100 independent examples each of four categories (pedestrian, automobile, bus, truck, street sign). For each example you will have at least 100 frames.
• Show the annotated sequences to Piotr

12/08/07

• Define tracking performance measure
• Test tracking on the annotated sequences

Logs on Meetings

2nd Term Meetings

02/27/08 Meeting canceled (Piotr wasn't free).

• Should reschedule to later this week

02/20/08 Meeting w/ Piotr

• Discussed how to improve the speed of the clustering process (use PCA)
• Discussed how to make the plots better (for ex. chi-squared distance to measure the difference between histograms)
• Discussed ways to improve the clustering results (use normalized patches -- 0 mean, unit variance, rescale, and etc.)

02/13/08 Meeting w/ Piotr

• Discussed the k-means algorithm -- when it succeeds (points of a cluster are grouped roughly in a sphere) are and when it fails (too few cluster centers, wrong starting points, etc.)
• Discussed ways to measure codebooks.

02/06/08 Meeting postponed due to a deadline Piotr had

• Piotr sent me detailed feedbacks for my mid-term progress report via email.

01/31/08 Meeting w/ Piotr

• Discussed the error measures. Piotr suggested better presentations of the results.
• Discussed the patch features. We'll try simple algorithms first, namely K-means, to see what results we can get out of these features.

01/30/08 Meeting was canceled (Piotr didn't feel well and wasn't in lab)

• It should be rescheduled by the Friday.

01/23/08 Meeting w/ Piotr

• Discussed error measures
• Discussed how to implement new feature spaces
  • Other color spaces like LUV, HSV. Considering only using 1 or 2 channels (despite brightness).
  • Gradient spaces:
    • 1 channel: magnitude
    • 2 channels: Gx, Gy; mag, orientation
    • 3 channels: magnitude, binned by angle

Meetings before 01/23/08

• Discussed where this project is going
• Discussed what to do this term

1st Term Meetings

No Meeting on 11/21/07 due to Thanksgiving

Meeting w/ Piotr on 11/14/07

• Discussed current work
• Discussed how to use some tricks to speed up the code(like << >> instead of multiply/divide). Set up an account on the Windows computer to label data.

Meeting w/ Piotr on 11/07/07

• Discussed current work
• Went through my code for kernel-based tracking (fixed scale version), and discussed how to improve the speed
• Discussed future work - improve the processing speed, add in adaptive scale feature, and label 5 hours' data to serve as ground truth for future benchmark.

Meeting w/ Piotr on 11/01/07

• Discussed current work
• Showed me how to use Piotr's program to label data
• Made plans for next week

No Meeting on 10/24/07 due to my GRE test on 10/26/07

Meeting w/ Piotr on 10/17/07

• Discussed future work
• Discussed what need to pay attention to when reading the paper
• Piotr set up my svn account, showed me how to use it, and sent me references

Meeting w/ Piotr on 10/09/07

• Set goal for this week: Read and understand "Mean Shift Analysis and Applications" and "Kernel-Based Object Tracking" (see references)
• Discussed density function and how to estimate it using sums of kernels
• Discussed how mean shift works in general
• Discussed how mean shift based filtering works and possible applications

Notes

11/11/07

• The color histogram is based on RGB values, so it's not invariant to changes in illumination. We can try to use YUV, where Y is the intensity, and UV contain the information about color. We could make Y to be adaptive

1. take a look at LUV / LAB color spaces LUV

11/10/07

• Note that the tracker uses a uniform histogram to represent a target or a candidate, but this is inefficient because it works the best only when the color distribution of an object is uniform; however, colors of an object often fall in a certain rage. For example, on human faces, only white, yellow, and black are the most commonly detected, so lots of bins in the histogram have little information. Using an adaptive scale histogram may work better - Use more bins (finer intervals) on the range where colors appear most (higher color density), and use less bins on the range where colors rarely appear on the object.

1. take a look at kd-trees

11/07/07

• Note that the kernel-based tracker isn't entirely orientation invariant for objects whose width and height are different, so if a target is rotated by 90 degrees, the tracker might loss the target. However, it has no problem if it is rotated by 180 degrees.

References

[1] D. Comaniciu, V. Ramesh, and P. Meer, “Kernel‐Based Object Tracking”, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 25, no. 5, May 2003.

[2] D. Comaniciu, P. Meer, “Mean Shift Analysis and Application”, IEEE Int. Conf. Computer Vision (ICCV'99), Kerkyra, Greece, 1197‐1203, 1999.

[3] F. Jurie, B. Triggs, “Creating Efficient Codebooks for Visual Recognition”, Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005.

Previous Work:

Fg/Bg Segmentation