Handling Videos

From Vision Wiki
Jump to navigation Jump to search

Update for Matlab 2009a

Matlab 2009a has added support for video in Linux which may make some of the advice below obsolete. Use the mmreader command to open a video file; this command returns an object. Then you can use I=read(obj,ii) to retrieve frame ii of the video. See the Matlab documentation for more details. Matlab's video support is built on top of gstreamer, which is a multimedia framework for Linux. As long as gstreamer's codec for a given format is installed, then that video format should work in Matlab. I've been able to open MPEG2, MPEG4 (h264), and Divx AVI formats. --Ryan 14:19, 24 Jun 2009 (PDT)

General Notice

The advice given on this page are the result of experience with video file in the SO(3) group (a.k.a. Serge Belongie's UCSD Vision Group) since 2003 as well as personal experience from V. Rabaud as an amateur movie maker.

Simply put, the advice given in here are known to be the best for the SO(3) group. They take into account the people we have, the hardware as well as the task we use movies for. The advice given will not be suitable for let's say movie industry, respect of exclusive use of Open Source software ...

This information has been reviewed and adapted by Claudio Fanti, in an attempt to enhance its relevance to the Caltech Vision Lab.

A few things about video formats

  • You have to make the distinction between a container format and a codec format. A ".mov" or ".avi" is not a movie format. It can contain whatever you want.

How to acquire

  • from a tape playable in a camcorder with Firewire/DV output.
    • We moved from a software (Adobe Premiere) to a hardware (FireStore FS-1) based solution. The FireStore allows you to connect a DV camcorder which plays the video, and transfers it onto an attached firewire hard disk. The compression is MPEG-2 (DVD quality) and the file is AVI.
    • In order to have a usable movie, you will need to deinterlace it.
  • from a webcam
    • Use the drivers given with it.
  • from a DVD
    • You can use the Gordian Knott package that is available on Doom9. You will find it pre-installed on the video.vision.caltech.edu windows computer.
    • Robot4rip will take care of extracting the video content from the VOB containers that are on the DVD. It will separate audio and video into an AC3 audio file and a d2v video file. It will (illegally) deal with most of the encryptions schemes used in DVD
    • You can then use the Gordian Knott software to read in the d2v file and select the video encoding options, the container (AVI, MSK, OGG), choose the destination audio format (MP3, AC3...) and run the job.
    • If you want to do some more advanced processing, then VirtualDub can do encode/filter/de-interlace/resize/etc of videos. It works well with AVI. It can be fed MPEG2 Videos (from the d2v container) thanks to AviSynth and its plugins. AviSynth has a large number of plugins that act on each frame to return a "processed frame" that can then be fed in the VirtualDub general framework for further processing.
    • The Doom9 page has all of the mentioned free software (mostly for Windows). A lot of useful step-by-step guides can easily teach you how to set the parameters of codecs for best performance.
  • from a video file
    • If you already have some video in AVI format you can try to use VirtualDub to convert it by opening it, and saving it with a new format/codec.
    • VideoMach is another option (we have an older version - free for educational purposes). It will take as input most file types and convert it to a different format. It can be used to collate a number of sequential images into a video, or split a video into images.


How to decode

  • Some formats can only be read by certain applications on windows: e.g. windows media player cannot read a ".mov" movie. In this case, you should use what is referred to as a "frame server". It is basically an application that will fool the application reading the ".mov" by being the intermediate between the reading of the ".mov" by quicktime and the media player. The best is [ http://www.avisynth.org/ Avisynth]. A simple example is here http://www.avisynth.org/YourFirstScript and the quicktime example is here: http://www.avisynth.org/QuickTime.
  • Some movie formats cannot be read on certain platforms ( e.g. Indeo is not supported on Mac OS X 10.4). Try first on windows.

How to encode

  • Concerning the format, it all depends on what you want to use your movie for: encoding a movie right is finding the best trade-off between movie size / CPU available for decoding / platforms on which it will be read /license
    • if you want to keep the full quality, keep DV. It is only 4 times compressed (hence 250MB/minute)
    • Divx (DX50) is a codec available on the main platforms (Windows, Mac, Linux) and quite widespread as it is used for DVD encoding by many people. It offers a good quality/size ratio and should be used to put movies on the web. It is based on MPEG4-part 2.
    • Microsoft MPEG 4 Ver 2 (MP42) is a codec available on the main platforms (Windows, Mac, Linux) and quite widespread as well. It is the best codec we could find that would reliably work with the MATLAB decoding library under Linux.
    • if you really care about the quality/size ratio but not about having everybody able to read your files, use h264 or Sorrenson (they are MPEG4-10) but this standard is still young

IMPORTANT : when using most MPEG4 (MP42/DivX, WMV9) codecs, try to apply a resize filter to your movie so that the image is a multiple of 4 (otherwise, you will not be able to encode or will get odd results)


Interlacing

  • everything you need to know about interlacing: http://100fps.com/
  • yes, many applications promise they can solve deinterlacing (e.g. Premiere) but the best we found (at least for DV), which is also the fastest, is the deinterlacing filter found in VirtualDub

Filters

  • VirtualDub and Avisynth, mentioned above, can apply some filters very fast (like cropping, color changing, resizing ...)
  • You might even find filters that do particular tasks (like de-interlacing via motion-analysis etc...).


Processing Videos on Linux in Matlab

The AviFile library is installed on our machines. It can open AVI files compressed in Microsoft MPEG 4, Version 2 (MP42) and will call a matlab script of choice (e.g., "ProcessMyFrame.m": INPUT: K, InImage OUTPUT: OutImage) to which the frame number and image data is passed as input. The (possibly processed) output frame that such script returns is saved in a MP42 AVI movie or a sequence of JPEGs, 1 per frame. I believe it can also open DX50 (DivX Version 5.0), with some sporadic issue, but will still output a MP42 file.

To get started, you can download this zip file which contains a "Test.m" main program, a "ProcessFrameTest.m" frame-processing script and the necessary ProcessAVI.cc mex-file that will interface MATLAB to the AviFile library.

Videos in Powerpoint

  • Powerpoint does not support certain video formats. Divx is one of them
  • to put your video in your .ppt, convert it using virtual dub using the codec windows media 9

http://www.microsoft.com/windows/windowsmedia/download/AllDownloads.aspx IMPORTANT : apply a resize filter to your movie so that the image is a multiple of 4 (otherwise, you will not be able to encode)

Misc

Retrieved from "https://visionwiki.caltech.edu/index.php?title=Handling_Videos&oldid=15815"