Download The Art and Science of Digital Compositing: Creating Seamless Images from Multiple Sources and more Summaries Art in PDF only on Docsity!
An alien spacecraft hovers over New York, throwing the entire city into shadow. Massive armies converge on a battlefield. A giant ape faces off against a Tyrannosaurus Rex. And the Titanic, submerged for decades, sails once more.
Usually the credit for these fantastic visuals is given to “ CGI ” (computer-generated imagery) or “ computer graphics, ” an attribution that not only broadly simplifies the technology used but also ignores the sizeable crew of talented artists who actually cre- ated the work. Computer graphics techniques, in conjunction with a myriad of other disciplines, are commonly used for the creation of visual effects in feature films. But the term “ computer graphics ” is broad and covers a wide variety of methods that rely on a computer to help produce images. Many of these methods are merely traditional methods that have been updated to take advantage of modern tools. In fact, even the typesetting of a book like this is now almost completely done using a computer, and as such this page could loosely be considered a piece of “ computer graphics. ”
When dealing with computer graphics as used for the creation and manipulation of images, we will usually break the subject down into two primary subcategories: 3D graphics^1 and 2D graphics. The names indicate whether the work is considered primarily three-dimensional in nature, or two-dimensional. The first category, the “ 3D ” work, involves creating a complete model of an object within the computer. This model can be viewed from any angle, can be positioned relative to an imaginary camera, and can generally be manipulated as if it were a real object, yet it exists only within the computer. Even though the way we are interacting with the object is still
Introduction to
Digital Compositing
C H A P T E R O N E
(^1) Do not confuse 3D imagery with stereoscopic imagery, a topic that we will also discuss in subse- quent chapters.
2 The Art and Science of Digital Compositing
based on a two-dimensional display device (the computer ’ s monitor), the model itself is a mathematical simulation of a true three-dimensional object. This model can be lit, textured, and given the ability to move and change. Once a particular camera view is chosen and the color, lighting, and animation are acceptable, special software will render the scene to produce a sequence of images.
While the 3D aspect of visual effects seems to get a great deal of recognition, it is only one piece of the puzzle that makes up a finished shot. The other half of the visual effects process involves working with preexisting images, manipulating and combin- ing them to produce new images. These images can be from just about any source, including rendered images produced by the 3D process. This process of manipulating existing images is identified as being “ 2D ” work because of the flat, two-dimensional images with which it deals and because there is essentially no attempt to introduce any three-dimensional data into the process. Not every film that makes use of visual effects will include 3D work, but any time there is a visual effect in a film, you can assume that 2D work was done. It is the backbone of visual effects work, and the fi nal, most important step in the creation of the desired imagery.
Even with a fully rendered 3D movie like Pixar ’ s The Incredibles , 2D effects and tools are used to enhance and integrate the rendered 3D images. 2D manipulations have historically been accomplished via a number of different methods, as we ’ ll discuss in a moment. But these days, most 2D work is done with the aid of computers, and the bulk of this 2D work is classified as digital compositing.
Definition
Digital compositing, as we are going to be discussing it, deals with the process of integrating images from multiple sources into a single, seamless whole. While many of these techniques apply to still images, we will be looking at tools and methods that are useful and reasonable for large sequences of images as well. Before we go any further, let ’ s come up with a specific definition for what this book is all about.
Digital Compositing: The digitally manipulated combination of at least two source images to produce an integrated result.
By far the most difficult part of this digital compositing process is producing the inte- grated result—an image that doesn’t betray that its creation was owed to multiple source elements. In particular, we are usually attempting to produce (sequences of) images that could have been believably photographed without the use of any postprocessing. Colloquially, they should look “real.” Even if the elements in the scene are obviously not real (huge talking insects standing atop a giant peach, for example), one must be able to believe that everything in the scene was photographed at the same time, by the same camera.
Figure 1.
An early composite photograph, Oscar Gustav Rejlander
’ s
The Two Ways of Life
. (Photo courtesy of the Royal Photographic
Society Collection, Bath, England. Web site www.rps.org .)
Introduction to Digital Compositing 5
Had the artist wished to capture this image on a single negative, he would have required a huge studio and many models. Even then, it is doubtful whether he could have lit the scene with as much precision or have positioned all the people in exactly the fashion he desired. Certainly it could have proven to be an expensive, time-consuming process. Instead, he painstakingly shot small groups of people and sets, adjusting each for the position and size that he would need them to be. In some cases, the only way to make them small enough in frame was to photograph them refl ected in a mirror. Once the various negatives were created, the combination pro- cess involved selectively uncovering only a portion of the printing paper and expos- ing the desired negative to that area.
The scene that resulted from this complex undertaking was designed to depict the two paths that one may choose in life. The right side of the image represents the righteous path, with individual figures who illustrate Religion, Knowledge, Mercy, Married Life, and so on. The left side of the image depicts somewhat less lofty goals, with figures representing everything from Idleness to Gambling to Licentiousness to Murder.
Photography was only just becoming accepted as a true art form, but Rejlander ’ s work was immediately recognized as an attempt at something more than the typical documentary or narrative photographs of the time. This is important to understand, since it points out that Rejlander used this combination technique in pursuit of a spe- cifi c vision, not as a gimmick. There was a great deal of science involved, but more important, a great deal of art.
While The Two Ways of Life received quite a bit of recognition, it was also the sub- ject of some controversy. Although part of this had to do with its subject matter (a Scottish exhibition of the work actually hung a drape over the nudity-rich left half of the image), the issue of whether or not such “ trick ” photography was ethical or artistically valid was continually raised. Eventually Rejlander himself denounced the practice, stating,
I am tired of photography for the public—particularly composite pho- tographs, for there can be no gain , and there is no honour, but cavil and misinterpretation.
Fortunately, the techniques continued to be used and refined even without Rejlander ’ s support, and few people today consider compositing (at least conceptually) to be par- ticularly dishonorable.
Motion picture photography came about in the late 1800s, and the desire to be able to continue this sort of image combination drove the development of specialized hard- ware to expedite the process. Optical printers were built that could selectively combine multiple pieces of film, and optical compositing was born. Many of the techniques and skills developed by optical compositors are directly applicable to the digital realm,
Introduction to Digital Compositing 7
Nowadays, optical compositing has effectively been replaced with general-purpose computer systems and some highly specialized software, but the concepts have not really changed. Before we start our discussion of these software tools, let’ s take a look at an example of one of these digital composites. We won ’ t go into a great deal of detail about this particular example just yet, but will initially use it to start presenting some of the common terminology used throughout the industry, as well as through- out this book.
Terminology
The example that we’ ll be dealing with is a scene from the feature film James and the Giant Peach , and is shown in Figure 1.3.
This particular composite was created from a multitude of different original images. We usually refer to the individual pieces from which we create our final composite as elements. Elements in this composite include the following:
● (^) The giant peach, shown as a separate element in Figure 1.4a. The peach is a miniature element, about a foot in diameter, and was photographed on a stage in front of a blue background, or bluescreen. ● (^) The giant mechanical shark, shown in Figure 1.4b. This element is a computer- generated image, built and rendered as a three-dimensional model completely within the computer.
Figure 1.3 A composite scene from James and the Giant Peach.
8 The Art and Science of Digital Compositing
● (^) The water, shown in Figure 1.4c. The water element is also computer-generated 3D imagery. ● (^) The sky, shown in Figure 1.4d. This element is a hand-painted backdrop (painted on canvas) that was photographed as a single frame.
Figure 1.4a The giant peach miniature in front of a bluescreen.
Figure 1.4b The giant mechanical shark.
10 The Art and Science of Digital Compositing
may be generated that are the result of some manipulation performed on an original element. Figure 1.5 shows such an element: a specialized image known as a matte that is derived from the bluescreen peach element and that will be used to selectively add the peach into the scene.
Figure 1.5 The extracted matte of the peach.
As you can probably tell, most of the elements that we’ ve shown have had some sort of additional processing performed on them as they were added into the final scene. Such processing, which might be used to modify the color of an image or the size of an element in the frame, is done throughout the compositing process in order to bet- ter integrate elements into their new environment.
You will commonly hear elements referred to as layers , since the various elements are layered together to produce the eventual image. Original footage that is shot with a camera and transferred into a computer constitutes a subset of elements usually referred to as plates. Thus, the peach element in this example would be referred to as a plate. Typically, a synthetic image such as the water would not be termed a plate, nor would any intermediate elements that were generated during the process of cre- ating a composite, such as the peach matte.
As you can see, there is little conceptual difference between the shot that was done for the 1933 version of King Kong and a digital composite such as the one we ’ ve just described for James and the Giant Peach (or in fact for the shot from the updated ver- sion of Kong that is discussed in Chapter 15).
Introduction to Digital Compositing 11
Organization of the Book
This book will attempt to cover a range of topics related to digital compositing, from some basic concepts dealing with digital image acquisition and storage to specific aes- thetic considerations necessary for producing a good composite. Initial chapters will provide enough background so that readers with only a basic knowledge of comput- ers will fi nd the book useful; in addition, there are a number of sections in this book that we hope even the most experienced professional will find useful.
The structure of a book such as this requires that the information be broken into well-defi ned categories. These categories (represented by chapters or sections within a chapter), while useful for organizational purposes, are ultimately somewhat arbitrary. The topics are all interrelated, and many discussions could easily have been placed into a number of different categories. To help identify related informa- tion, you will often find cross-references to relevant material that is located else- where in the book. The specific breakdown for how the chapters are organized is as follows:
● (^) Chapter 2 describes characteristics of cameras and vision that must be understood in order to produce realistic, believable composites. ● (^) Chapter 3 presents an overview of how images are represented digitally, including some discussion about the process of converting images from other sources into a digital format. ● (^) Chapter 4 covers some of the basic manipulations that are possible with digital images. These include using image-processing tools that are designed to modify the color, size, and placement of elements within a scene. ● (^) Chapter 5 takes this topic to the next step, with a look at the process of combining images or image sequences. This chapter is where the concept of a matte—an image that is used to selectively control certain combinatorial operations—is first introduced. ● (^) Chapter 6 deals with these matte images in much greater detail—particularly with the methods that are commonly used to create and modify them. ● (^) Chapter 7 is dedicated to concepts and techniques relating to imagery that changes over time, and to image manipulations that do the same. ● (^) Chapter 8 looks at tracking, the process of analyzing and duplicating the motion of existing objects in a scene. ● (^) Chapter 9 discusses a variety of methods that can be used to interact with the data, images, and software that you will be using. ● (^) Chapter 10 is a broad overview of the various formats that might need to make use of digital compositing, from film to video to multimedia.
Introduction to Digital Compositing 13
- Chapters 4 and 5 concentrate on specific tools, and as such may not be as relevant to someone who will not be involved with actual image creation. The section in Chapter 5 that discusses the concept of a matte is probably the only “ must-read” section.
- Chapters 6, 7, and 8 should be skimmed to get a basic idea of the concepts presented, but probably not much more than that is necessary. Chapter 9 is even less interesting to someone who will not be using any software directly.
- Chapter 10 should probably only be looked at if there is a desire to understand a particular format in more detail. This may come up on a case-by-case basis.
- Chapter 12 is specifically targeted to people who will be involved in the actual photography of the elements that will be used in a composite.
- Chapter 14 is mostly composed of advanced topics, but the end of the chapter does touch on a few disciplines related to compositing that may be useful.
- The final section of the book is the glossary. Although it is not something that should be read from beginning to end in one sitting, it should prove to be a valuable resource as one encounters unfamiliar terms.
Whether or not you actually have any formalized artistic training that should not deter you from learning the specific artistic skills that a good compositor (one who creates composites) uses on a daily basis. Part of this process involves learning the tools that are necessary. Just as a painter needs to understand the workings of pig- ments and canvas, a digital artist needs to have a technical understanding of the software involved. It is certainly possible to be an effective compositor without fully understanding all the technical issues that will be presented in this book. As with any other artistic discipline, experience and instinct can go a long way toward producing the final result. However, a basic understanding of the technology behind the tools can greatly improve the efficiency and problem-solving ability of even the best artists.
The rest of the art will come only with experience, although you will probably find that you are already well on your way. This is because you, and every other sighted person, have already spent a lifetime learning what reality looks like. The information may not always be consciously accessible, but the expert subconscious is surprisingly skilled at noticing when an image appears artificial or incorrect. This is both a bless- ing and a curse, since everyone else who will view your work will be similarly able to detect these problems. However, the ability to determine that there is a problem is the largest part of the process of realizing why the problem exists and learning how to fi x it. Throughout this book we’ ll certainly try to point out the scenarios in which visual problems can arise. These problems will often involve well-defined artifacts— undesirable items in an image that are the result of the process used to create the image. But compositing problems can go far beyond specific identifiable glitches. Greater difficulties arise when trying to accurately duplicate the myriad of visual cues that would be present in a real image. These are the things that the subconscious is so
14 The Art and Science of Digital Compositing
much better at perceiving, and this is why someone who creates composite imagery must become an artist. The same knowledge that a painter uses—from the interaction of light and shadow to distance and perspective cues—will be necessary to create a believable digital composite.
There is one final piece of business to be dealt with before we can continue to the rest of the chapters in this book. A disclaimer:
Different people, countries, and software packages do not always use the same names to refer to particular tools, operators, or subjects. In addition, due to the need to simplify certain things, just about every statement we make could probably be countered with some exception to the rule.
This statement isn’ t given in order to protect the author from any mistakes he’ s made within the pages of this book, but rather as a warning to anyone just starting to learn about this field. In many cases, you will find similar or identical concepts referred to by completely different names, depending on what book you are reading or to whom you are talking. Digital compositing is still a very young, volatile discipline. As such, there are many areas within it that have not become terribly well standardized.
A glossary is provided at the end of the book that attempts to cover a wide range of terms as they relate to digital compositing. Throughout the body of the text, terms being introduced are printed in bold to indicate that you will find a definition in the glossary. There are certainly places in the book where we have made some arbitrary decisions about a particular term and its usage. It is hoped this will help, rather than hinder, the process of terminology standardization within the industry.
