A Look at xRez Studio's Gigapixel Panoramic Photography
By John Virata
xRez Studio, a consortium of visual effects artists has introduced a type of visual effects based on gigapixel panoramic photography. The group, headed by former Digital Domain alums Eric Hanson and Greg Downing, has refined a photographic process that generates high quality image resolutions of up to 150,000 pixels wide, which they say is larger than any film standard, including IMAX. They have been using gigapixel imaging in visual effects for feature films as well as other applications, such as natural park interpretation. The imagery at xrez.com is really compelling, as is the nature in which they create the imagery. DMN interviewed Eric Hanson on how images obtain their gigapixel specification and also details the use of Gigapixel Panoramic Photography in several films.
DMN: What is gigapixel panoramic photography and how many images would constitute a gigapixel image?
Eric Hanson: Gigapixel imagery refers to the amount of pixels or effective detail in an image, gigapixel simply meaning containing over 1000 megapixels. A standard digital camera produces around 10 megapixels, but one of our images typically contains around 100-800 times that amount of resolution. We create these massive images by using a motion controlled camera rig, which can manage the fine precision and tedium required to shoot up to 800 individual, carefully overlapping images. The images are then stitched together using specialized software to consolidate the array of shots into a single image.
We shoot both what we term "high density narrow field" images which use very long lenses to maximize the potential gigapixel data into a standard field of view, as well as creating spherical panoramic shoots, capturing an entire 360 degree environment, which spreads the high resolution equally throughout a sphere around the camera. Either offers a wealth of resolution, but the narrow field is used when we would like extreme exploration of the image, whereas the spherical image can be used as the basis for virtual cinematography and visual effects shots.
DMN: How does a visual effects artist/3D animator integrate xRez gigapixel panoramic photography into their workflow? Does it work as a plugin to the existing 3D package or is it an ordinary image file that has been enhanced?
EH: Our work with gigapixel images came out of a desire to capture a variety of commonly used shooting locations around LA in high resolution for digital backlot purposes in visual effects.
Our experience stems from creating digital environments in the visual effects field, and we believe this technique offers unique opportunities for effects productions. The gigapixel images themselves can provide reasonable substitutes for scenic backings or even live action shooting or in some cases, but the real opportunities exist in unifying the gigapixel data with the 3d geometry of a scene. Using DEM terrain data, photogrammetry, or LIDAR scans, one can establish rich 3d re-creations of existing locations that provide the starting point for intricate matte paintings or shots compositing CG characters or elements into the scene. The techniques use common pro 3d applications, but we have developed unique techniques and proprietary tools for doing the integration.
|Click for full view|
DMN:What are the camera requirements to capture the gigapixel resolution images? Are the requirements available to those who wish to work with gigapixel imagery or is this a service provided by xRez Studio?
EH: The camera rigs we use are off the shelf pro SLR gear, but we have recently shot w/ the 39 MP medium format Hasselblad H3D in order to speed the capture or even raise our ultimate resolution limit. Gigapixel imagery can be approached by many, but we have taken years to perfect and standardize a production-ready methodology and production pipeline in order to offer the work professionally. It is a process fraught with potential oversights, but we have refined our process to the point of completing over 200 gigapixel images for a recent client. We offer a range of services, including consulting, shooting, licensing our image library, to entirely producing VFX shots for various productions.
DMN: What kind of processing power is required to work with and manipulate these types of images?
EH: We rely on the fastest hardware available, using 8-processor 64-bit machines w/ 8 GB of RAM, but of course the magnitude of the images devours whatever we use. It is a testament to Photoshop that the images can be handled at all, but with fast machines the workability is reasonable. Processing demands scales up accordingly, however, so images are handled a bit more conservatively during editing. We have developed custom techniques for importing the gigapixel image into Maya, which would normally prohibit such image size. We have a 30 processor render farm we have re-termed a "stitching farm" to handle the stitching load, which can take up to 2 days of render time per image. Disk space is also an issue, but we have 26 TB in our facility to handle large jobs.
|Visit xrez.com for full panoramic images.|
DMN: Does xRez provide the panoramic stitching software?
EH: We use commercial stitching software known as PTGui, which is one of the few apps that can handle gigapixel images and produce excellent output, with no degradation from the original photography. We have written some custom scripting to provide fast transition between our motion control hardware and stitching software. We are also developing code to easily align DEM terrain data with gigapixel images within Maya.
DMN: In addition to Magnificent Desolation, what other films can xRez-based gigapixel panoramic photography be seen?
EH: xRez was the VFX vendor for a recent documentary on the Wright Brothers, where we used panoramic images to build 3d digital backgrounds for the CG Wright aircraft to fly within. We have other projects beginning this fall which are more based on scientific visualization, to help describe the geologic history of Yosemite Valley and explore the marine biology of Santa Catalina Island. This fall, we also plan to introduce our library of 200 gigapixel urban images we have acquired this year, which should be very useful to the VFX field.
DMN:What type of planning was required to make the shot in Magnificent Desolation?
EH:That shot needed to describe a deep trench on the moon called the Hadley Rile which was never explored from within. The Apollo 15 astronauts had shot hundreds of images of the rim panoramically w/ Hasselblads, overlapping the shots as we do today, which provided a great base to stitch from. Once these panoramas were assembled from a few vantage points on the Rile, we employed photogrammetry to faithfully extract the 3d geometry of the canyon. A camera path was then established, and set dressing of moon rocks and sand was built around it. Finally, a series of 24 overlapping matte paintings were camera projected onto the geometry, which provided an efficient way to recreate a photoreal sense of the canyon. A stereo camera rig was used to render into the final stereo Imax format. This shot proved to be a great testament to the strength of merging panoramic photography with photogrammetry and visual effects.
DMN: What movie or movie scene employing traditional effects methods might have benefited from using xRez techniques and gigapixel panoramic photography?
EH: Our techniques are a logical extension of the "pan and tile" technique developed originally at our former employer Digital Domain. Many major features at several VFX houses have utilized large format or stitched photography as a base for constructing backgrounds, but we can now offer unprecedented resolution, allowing new creative freedoms and opportunities, especially with our unique integration of 3d geometry and photogrammetry.
Gigapixel imagery is in its infancy, and it will be exciting to see the creative uses unfold once it reaches mainstream use. We hope to help pioneer some of that ourselves!