You are viewing notbrainsurgery

This · is · Not · a · Brain · Surgery


Poor Man Video Thumbnail Generation

Recent Entries · Archive · Friends · Profile

* * *
If you are building yet another YouTube-like service, eventually, you will be facing the problem of generating thumbnails for video clips. The simplest solution would be to get some random or fixed frame, and use is a thumb. Unfortunately, this does not always produce best results. You can try to implement more sophisticated algorithms, like described here (Quote: "The shot and key frame are selected based on measures of motion and spatial activity and the likeliness to include people. The latter is determined by skin-color detection and face detection."). I did not have time and patience to implement such complex algorithms, so I came up with one of my own, which is really simple, could be implemented in couple of hundreds lines of code and works pretty well.

The main idea is very simple: we analyze first few seconds of a clip and build histograms of color distribution per frame. Then, we average them, building an averaged color distribution histogram. Then we find a frame, which is closest to the average value (I am using RMSE to estimate "closeness"). We select a frame close to beginning of the video, which makes selection process faster (less frames to examine) and less likely to include spoilers. Selected picture is similar in color distribution to the overall video theme, making it more likely to display typical frame.

I run it on few hundreds video clips, and it shows pretty good results. Of course, these results are not representative, I've selected most interesting ones but generally I think it is very usable. You can grab the source code and try it yourself.
* * *
* * *
[User Picture]
On December 29th, 2006 04:57 pm (UTC), uncle_ziba commented:
Hmm - interesting. This probably finds an average frame but not necessarily the key frame. Try this test: pan across the sky for 3 seconds then focus on a bird in flight for 2 seconds then pan across blank sky again for 3 seconds. would your algorithm find the frame with the bird?
[User Picture]
On December 29th, 2006 05:24 pm (UTC), notbrainsurgery replied:
Since in this hypothetical video you see the sky for six seconds and bird only for two, arguably the frame with the sky is better characterize this video. I could not know whether the bird was the object of your movie. It might as well being about nice cloud formations and bird gets into the frame by mistake.

But I agree, this algorithm is not perfect. But in a lack of more sophisticated solution, this is better that selecting random frame.

* * *
[User Picture]
On January 2nd, 2007 12:24 pm (UTC), andy_scott commented:
привет крокодил
и с новым годом
* * *
[User Picture]
On March 4th, 2007 07:29 am (UTC), vzaliva commented:
This algorithm is currently used at http://umundo.com/
* * *
[User Picture]
On April 14th, 2007 05:42 am (UTC), notbrainsurgery commented:
Also used in ImageShack toolbar.
* * *
[User Picture]
On May 1st, 2009 01:05 am (UTC), vzaliva commented:
ffmpegthumbnailer
Also used in

ffmpegthumbnailer.

* * *

Previous Entry · Leave a comment · Share · Next Entry

* * *