Panotour - User tips - Livepano workflow
Contents
Introduction
This is a workflow guide to shooting and rendering a LivePano, based on users experience. It is not meant to replace the Kolor documentation, nor does it deliver 'golden rules'. I just recorded these guidelines as an aid for myself.
Basic knowledge of shooting, stitching and rendering a panorama as well as experience in the use of Autopano Giga and Panotour Pro are assumed if you want to be able to follow this guide.
-Tutorial by Pierre Gielen, 10 july 2014
virtualtours.nl
STEP 1: Hardware Setup
Use a DSLR that can record full HD video (1920x1080). I have tried my trusted Nikon D90 which has half HD (1280x720) video; the results were disappointing.
Use a tripod, a fisheye lens and a nodal adapter for both the recording of the still images of the panorama and the video. Changing the lens in between shots means asking for trouble.
In noisy or windy conditions, use an external shotgun microphone which can be attached to the flash mount, with a hood or fur ('dead cat') to eliminate wind noise. If all else fails, use a Lavalier microphone, possibly wireless. Investing in a balanced (XLR) microphone system sounds like overkill, but it can be of use in situations where long cables pick up too much radio interference.
This is the equipment I use:
- a Nikon D7100 with Nikon 10.5mm/f:2.8 fisheye lens.
- A Nodal Ninja 3 MKII.
- A T-Bone CM-400 shotgun microphone (from thomann.de), which is affordable and small enough to not show up in the images.
- An Audio Technica Lavalier microphone with 6m of cable as a backup for noisy surroundings. The cable however has to be removed from the video using Adobe After Effects (or similar software). In the future, I would prefer a wireless Lavalier microphone.
STEP 2: Shoot the Panorama
First: don't forget to turn off your cell phone! It may cause a lot of interference in audio recordings.
Set up your camera and nodal point adapter on a tripod using a fisheye lens as usual. Connect the microphone and test if it works.
Position it for the recording of the video as precise as possible. Make some test shots to see if the 'actor' is well lit and looks good.
Record the video. Make sure the actor, if moving, stays within the frame!
Also important: if the sun is low, check if the shadow of your actor falls within the frame. A cut off shadow will mean you have to edit the video in order to remove the shadow entirely. That should be a last resort, because the result will be unrealistic.
Preview the video on the camera using headphones to listen if the sound is okay (I have had frustratingly disappointing results with good video and unusable sound, which I only discovered after returning to the office; you don't want that to happen!).
After you have recorded a usable video, without moving the camera or adjusting anything, take all still pictures you need for a normal panorama.
STEP 3: Stitching
Put the stills and the movie in one directory. Open the movie with VLC Media Player (www.videolan.org). It is useful and free software that you may already have.
Pause the video on a typical frame and take a screen shot using the menu Video > Take snapshot. It will save a screen shot named vlc****.png in your Pictures directory (on Windows). Move it to the directory where your stills and movie are located, just to keep everything together.
Open the directory where the stills and movie are located. In the group window, click on the group settings icon. A window opens. Now make the following changes:
Under Detection, select 'Force every image to be in one panorama' and set Detection quality to High and control points to 100 (default=50).
Detection will take longer, but this is necessary because of the differences between the vlc file and other images.
In APG 3.5 and up, you will have to click the select box next to Detection (automatic) to adjust this and you can also set the filtering model to Spherical (complete model on the sphere). Next, adjust the optimization settings as follows:
For APG 3.0:
- Set Focal calculation to Force different
- Set distortion calculation to Force different
- Set distortion model to 2nd order + optical axis center.
- Deselect Cleanup control points or links.
Click OK.
For APG 3.5 and up:
Leave the optimization settings at their default values (being automatic, mostly) unless the results come out poor. If that gives you bad results, make the following changes:
- Select Optimization stages (manual)
- Deselect bad points and bad links
- Select Optimization scopes (manual)
- Grouping criteria: Has EXIF
- Yaw Pitch Roll: image
- Focal: Automatic / group
- Distortion: Optimize 2nd order / group
- Offset: Automatic / group
- Viewpoint: Do not optimize
Click OK.
Finally, in the Panorama tab, under Projection and crop, set Default crop to Maximum projection range. This makes sure the outcome will be a 2:1 panoramic image even if no complete panorama is detected. Without the 2:1 dimensions, PtP will refuse to run the Livepano plugin!
Also in the group windows, click on the image properties icon. The window image properties opens. Select the vlcsnap image an set its Focal to the same value as your fisheye lens and its Lens type to Fisheye. Click OK.
Click on Detect. After detection is finished, click on the Edit tab of the panorama. Normally, RMS will be okay as you are used to when making panoramas, but this time, it is possible that the vlcsnap image does not have links, because it's orientation, size, color and contrast differ from the other images. If this happens, we'll have to correct this manually, as follows:
- In the panorama editor, open the control points editor, and (automatically) add control points between the vlcsnap screen shot and all other images that overlap with it. There will be lots of red flagged control points. Don't worry about them, just keep adding control points. When you're done, click the Optimization icon (not the Quick optimization icon, that doesn't do much) and most of the control points will turn green. Close the control point editor.
- In the Panorama editor window, perform the usual actions to improve your panorama, like correcting the horizon and tagging ghost objects for deletion. Finally, go to the images list (N-0) and deselect (but do not delete!) the the vlcsnap image. If you do so, it will not be included in the rendered panorama (this being a sort of a 'background' for the video, but its control points will be saved in the .pano file, so the video can be precisely positioned in the background.
Save the .pano file in the directory where the stills and movie and screen shot are located. Render the panorama to your own preferences. My preferred settings for a LivePano are: width 8000, height 4000 (to keep things small), interpolator Spline36, Blending preset Anti-ghost, format TIFF 8 bits or JPG, resolution 72 DPI.
STEP 4: Rendering
After rendering, go to Panotour Pro. Create an empty project. Choose File > Add panorama and add the rendered image. Give the project a name. Use your own preferred settings.
Don't start the tour with a Little planet effect, since the video will start immediately after loading, it does not wait for the effect to finish! This means you will hear the audio before you see the video.
In PtP, go to the Style tab and scroll down in the left bar to Video > default video spot style. You can set the video to not loop when reaching the end as to not drive your audience crazy, and deselect directional sound (both are on by default). My personal preference is not to use directional sound which forces the user to keep looking at a static picture of a talking person, when the whole idea of a panorama is to look around and still get all of the information you need. But then of course, it's not always about people talking. You may want to add a video of an animal or an inanimate but moving object.
Choose File > Import > LivePano to start creation of the live panorama. In 5 screens, your LivePano movie will be rendered.
Screen 1/5: Panorama Selection
- If there is more than one panorama in the list, click on the one you want to add video to.
- In the input box Autopano .pano file, add the location of the corresponding .pano file.
- In the input box Video file, add the location of the corresponding video file.
Click Next.
Screen 2/5: Video Frame Selection
- Select the video frame (vlcsnap****.png)
Click Next.
Screen 3/5: Final Video Hotspot Area
- Turn the image 90 degrees left.
- Set the maximum size of the area your actor moves in.
Click Next.
Screen 4/5: Blending Mask
This is a critical step for invisibly blending your video into the panorama, so maximize the window.
- Using your mouse, draw a wide polygon around the actor, if possible following straight lines.
- Click space to complete and close the polygon.
- In the selection box on the right, select Display blend (Panorama + video).
- Brightness and color will probably be off. Use the contrast, brightness and gamma sliders to adjust the video imago so that it can seamlessly lend in. This is not always easy. Videos tend to have a higher contrast and a smaller dynamic range than still pictures, especially in high contrast situations and certainly if you're shooting the panorama in hdr. Try to minimize the differences on sight.
- To avoid sharp borders between video and panoramas, also soften the edges of the polygon in this screen.
Unfortunately, there is no way to save your settings in this stage and undo them or adjust them in a later stage. If anything goes wrong, you'll have to repeat all the 5 stages in LivePano again, making experimentation very time consuming.
Click Next.
Screen 5/5: Render
- Set the flags on the start and end positions of the video clip you want to include.
- Set the fade in and fade out time (500ms is a good value) for the start and end of the animation.
There is probably a problem here with synchronization of video and audio, since LivePano does not offer the possibility for cutting exactly on a keyframe. I have noticed that the end result is not always lip sync, even if the input video material is. Kolor may want to look into this for a next release.
Click Render.
LivePano now renders a high and low resolution version of the video. This will take several minutes. For 25 seconds of video, my laptop takes about 20 minutes... Am I in need of a faster computer? Definitely, the faster the better, but I do not expect the performance to improve so drastically that it will take less than a minute. LivePano may be in need of a 'quick render' function just for editing.
Final Notes
Here you can find a sample of a LivePano shot recently and produced following these guidelines. In Dutch, I am saying that a LivePano gives you an unique opportunity to send your message to an audience that is not willing to read long articles. It's not perfect (yet), for one thing, it suffers from the mentioned synchronization issue.
I showed it to my wife and she said it was not good at all. So I admitted that the syncing could be better and that I would have used an artificial light source on the subject (being myself) for the video because of the sunny and shadowy places if that wouldn't have interfered with the panorama it had to be blended with. To which she answered 'I don't know what you're talking about, but you shouldn't stand in such a way in front of the camera, it looks as if your trousers are too short and your socks show.'
There you have it. You can follow a step by step guide, but making a LivePano is an activity in which you should keep in mind a hundred and one things, and more... Thus giving you the chance to show your craftsmanship!
BACK TO: Documentation / Panotour Documentation