Fun : Stitching video frames
Contents
Introduction
Traditionally, using Autopano to create panoramas involves two major steps: the photo shoot and the stitching.
The first step seems to be absolutely necessary; without it, how do we get images we can match up and make into panoramas? There is an alternative, however... Video!
Why not give Autopano video images to stitch? When a camera moves, a scene dissolves into a series of images than can be matched up. The resulting panorama can be quite interesting.
Here’s an example of some (sadly) famous footage that could be used for this type of project:
  |
This opens up a new world: no longer do we CREATE panoramas from nothing; we can FIND panoramas hidden in videos!
Using videos in Autopano sounds like a fun idea, but, you’ll say, there’s a problem. Autopano can only handle images, not video.
Of course, so let’s give it images, and I’ll show you how to extract frames from a video using a free, high-powered tool called FFmpeg.
How to extract images
FFmpeg is a command line tool available for all platforms whose main purpose is to encode and decode video flows and convert them.
However, it is also capable of performing more exotic operations like extracting frames from a video.
The official FFmpeg project site has very thorough documentation for Windows and MacOSX users. The commands discussed in this tutorial are functional on Linux.
We have a video entitled jfk.mp4 which runs at 25 frames per second. Autopano doesn’t need all the frames; one per second of film will be quite enough. Here is the command line, explained:
ffmpeg -i jfk.mp4 -r 1 frames/jfk-frame-%03d.jpg
- -i jfk.mp4: input file
- -r 1: option specifying the number of frames per second (one in this example)
- frames/jfk-frame-%03d.jpg: template for filenames to be generated
Image output files are placed in the previously-created directory frames/.
jfk-frame-%03d.jpg is a name template which tells ffmpeg how to name the images that are generated and which format to encode them in.
Here the images will be in jpeg format and will be assigned 3-digit numbers including 0 (which is useful for sequencing them). Here are the contents of the directory once the operation is finished:
And there you go! Note that each jpeg file includes the image’s position in the video in seconds, which can be useful later on. (This is of course linked to the value assigned to the -r option in ffmpeg.)
For long videos, the name template will need to be changed, replacing %03d (up to 999 images) with %04d (up to 9999 images, or 2h46m39s at one image per second), or with greater values if necessary.
Finding Panoramas
Now that we know how to extract frames from a video, we still have to find panoramas. There are two possibilities: either you’ve located a sequence in a video that would make a nice panorama, or or you would rather let Autopano find one for you, something it does very well, as we shall see:
- In the first example, I’ve located a section in the video jfk.mp4 between 19s and 34s that seems suitable for a panorama. I drag all the files from jfk-frame-19.jpg through jkf-frame.jpg to Autopano and stitch a panorama.
- In the second example, all the frames are extracted from a video and placed in a file, and the file is dragged to Autopano. Detection finds various panoramas and classifies them by the number of images they contain.
With this method, most interesting panoramas generally end up at the top of the list. The screenshot below shows the product of 419 frames taken from an archival film of the Second World War and input in Autopano for detection.
Note : Some panoramas are the result of a slight camera movement and are not at all interesting. On the other hand, some reveal particularly intriguing scenes.
 |
Panoramas from before we knew how to make them
The example in the preceding paragraphs illustrates an idea: Extracting panoramas from videos made in an era when panoramas did not yet exist.
Here are some interesting panoramas where historic events can be seen as never before!
 |
Obviously, the quality of the rendering is not at all comparable to panoramas created from photos taken with modern equipment. But it should be understood that the interest of historical panoramas does not lie in the quality of picture, but in rather in the visual content that has never been published before.
Here are some panoramas created from a documentary film integrating footage from the Second World War (source: http://www.youtube.com/watch?v=lgS6gnyOOlE).
 |
 |
 |
Panoramas of places where no man has gone before
No one has gone to Mars yet to take panoramas. However, with Autopano, we can make good use of photos and films taken by the Mars Rover. The following film is especially interesting for our purposes:
 |
Using the principal explained above, here is the panorama created from the video:
 |
Closing film credits
The movie ends, the credits roll, and you can’t quite read it all. Don’t worry, you’re normal! If you’ve recorded the movie, you keep hitting pause/play, rewind...and getting frustrated. Those days are over!
Here is an excerpt from the first “Pirates of the Caribbean” film. The frames were extracted and Autopano was used to assemble them.
All you have to do now is print it out on a parchment scroll and unroll it at your leisure.
 |
Sports: A play unfolds
A panorama taken during a sporting event will always be a picture of a given moment. It would be very difficult to take a picture of an entire play. That’s exactly what video is for. What happens if we apply the above method of photo stitching to a sports video?
Let’s take one of Zinedine Zidane’s best-executed goals. Here’s the recap:
1. The goalie clears the ball, which is then deflected off a player’s head
2. Zidane kicks the ball on the fly at 35 meters from the goal
3. The opposing goalie is tricked
 |
The panorama created from the video merges 3 phases of the play:
 |
Conclusion
This tutorial presents a new use of Autopano: extracting images from a video. These images are usually of much lower quality than photographs, and often prove to be blurred.
It is never possible to make them into panoramas rendered perfectly down to the last pixel, even with HD quality film. Their primary interest lies mostly in the content that can be discovered, as we have seen.
You’ve just learned how a new world is opening up, limited only by your imagination. It’s time to dust off those old movies from your vacation and begin your quest for hidden panoramas...Have fun!