Semiotic Review 9: Images - article published April 2021
https://semioticreview.com/ojs/index.php/sr/article/view/71

A Peircean Lens on Cinematic Special Effects

Martin Lefebvre
martin.lefebvre@concordia.ca


Abstract: The aim of this paper is to reconsider how Peirce’s conceptions of iconicity and especially indexicality can help us logically account for some of the issues that pertain to representation in the cinema. To this end, the study centers on the practice of special effects as a way to reground the entire area of thinking about indexicality within the study of moving images. The author wishes to show how Peirce’s concepts not only remains viable for thinking about cinema but also offer a powerful tool for critically analyzing images.

Keywords: indexicality, iconicity, Peirce, special effects, cinema


This paper uses Charles S. Peirce’s semiotic concepts of iconic and indexical signs to examine forms of visual and perceptual deceit enacted by visual and special effects in films.1

Visual and special effects are deeply semiotic phenomena: they seek to represent something by way of something else. In many instances, however, they also seek to conceal the process whereby something is made to stand for something other than itself, wherein lies their deceit. This is especially striking when the fictional world that special effects seek to depict is congruent with the real world. As I will endeavor to show, Peirce’s semiotic offers sufficient flexibility to describe the technically simple but conceptually complex and layered forms of representation at stake in this process. The pages that follow require, however, that we move beyond the more basic—some might say “impoverished”—understanding of iconicity and indexicality commonly found in film studies, among other fields.

Thinking about cinema is often confused because of a difficulty in distinguishing the indexical from the iconic. This is most obvious in André Bazin’s famous “Ontology of the Photographic Image” essay, where he writes of the photographic image that it offers us “the object itself” and that “it is the model” (Bazin 2009:8). The idea has dumbfounded many a film theorist over the years (see Morgan 2006). However, from Peirce we know that iconic signs are indeed indistinguishable from their objects in the quality that they share. Therefore, like all icons and iconic signs, the photograph and its object do share the same identity. The mistake of Bazin’s interpreters—and perhaps, even, of Bazin—was to think that such is the case only or especially for photographic images, and that this is in particular due to their nature as “automatic imprints”—that is, as indices rather than icons. As we will see, special effects in live-action film throw this misunderstanding into relief because of the way they interfere with how fiction meshes with reality as the image knots together indexicality (in two distinct forms) and iconicity.

1.

It is a truism to say that digital cinema has revolutionized how films are made. The entire workflow has been transformed in important ways and visual effects have now become ubiquitous. There’s hardly a film—or even a single shot!—made today that doesn’t use them for minor touch-ups or else to create entire virtual sets composed from hundreds of elements, either photographic or CGI, into which actors and props, initially shot against a green screen, are digitally composited. As Tanine Allison (2016:185) argues, “since digital technology revolutionized filmmaking in the last decade of the twentieth century, visual effects have never been more important; rather than being an afterthought, an extraneous or inessential piece of the puzzle, they are at the very heart of contemporary visuality.” In many ways, the principles of pre-digital effects are still in place—composites, whether digital or employing glass paintings, still seek to create an impression of spatial contiguity by combining heterogeneous source elements—yet they now possess an unprecedented flexibility. The result is that filmmakers have gained an almost complete plastic control over the image. And because many of these digital effects are seamless, they are absolutely invisible, as long as they don’t challenge a number of obvious constraints.2

As far as film scholarship is concerned, the digital turn, and the exponential growth of visual effects which has accompanied it, has offered a new opportunity to revisit classical film theory as well as some foundational ideas about cinematic representation. Among the issues that have resurfaced is a concern for cinema’s indexicality. Although, most will now readily concede that digital cameras don’t rob images of their indexical value, since like their predecessors, they offer a record or trace of whatever stood in front of the lens. Visual and special effects, however, offer a different kind of challenge to indexicality—though not necessarily a new one (except for their sheer volume and the flexibility they entail). To see this requires us to first reflect on the very status of cinematic fiction (as I do in this section) and on a fuller discussion of iconicity and indexicality in film (as I do in the next section).

Most of the earliest films by Edison or Lumière offer us cinema’s “zero degree”—a sort of baseline—where the world of existents is captured by the camera in a single take and where there is no distance, no gap,3 between it (as it exists independently) and the “film world” (as it appears on screen). With fiction, however, a wedge gets driven between the real world recorded by the camera and the film world we are shown. We can call it a paradox that cinema can show us fictional film worlds by using recordings of the camera-independent existing world masquerading as imaginary. The reason for calling it such is that two distinct universes of reference (what logicians also call universes of discourse)—the real world that lies in front of the camera and the fictional world it is meant to depict—share the image in such a way that they may be understood to overlap or be superimposed upon each other.

For example, when Lauren Bacall and Humphrey Bogart embrace in To Have and Have Not (Hawks, 1944), these two worlds are intertwined as both actors and characters kiss on screen.4 To say that there are two distinct universes of reference here means that two statements such as “Bogart and Bacall are kissing” and “Harry Morgan and Marie Browning are kissing” can both strike us as true descriptions of the image though only in as much as they direct our attention or point to two sets of “objects” that don’t share the same worlds, and whose “existential-status” are different: the names “Bogart” and “Bacall” are used here as indexical signs pointing to individuals who exist (or have existed) in the world in which we live, while those of “Harry Morgan” and “Marie Browning” index fictional individuals belonging to a fictional world.5 Visual and special effects are usually understood as further dividing these two worlds of reference, to the point where it would seem to make little sense to speak of indexicality when the image we see—save perhaps for a few details—has been completely constructed, digitally painted and composited in postproduction. In fact, the claim made that digital cinema is more like painting or drawing than photography seems to only find its full realization when visual effects are taken into consideration. So where does this leave us with regards to indexicality?

Tom Gunning has argued that indexicality—by which he means the “diminished way” this concept has been used in film studies—“may not be the best way, and certainly should not be the only way, to approach the issue of cinematic realism” (Gunning 2007:21). This is absolutely true, especially since realism can be found in various other means of representation—such as painting or literature—that don’t rely on anything like a direct trace or imprint of what they depict the way photography does. And yet, regardless of realism, both can represent using indices. The fact is that realism and indexicality should be understood as entirely separate issues. In light of this, it seems to me that what is sorely needed, what visual and special effects require of film studies, is that we come to clarify as much as possible the questions raised by these terms.

To do this, we need first to go back to the source of indexicality, to the work of American philosopher Charles S. Peirce and consider the basic role that he saw indexicality as serving in representation. To put it simply, the index is a concept developed by Peirce to answer the problem raised by the representation of particulars or individual existents, that is to say objects such as they can be experienced or observed; a problem itself arising in the context of Peirce’s development of a realist epistemology. An individual existent is something that manifests its being in its resistance or its reactivity, it possesses haecceity, a “thisness,” that Peirce saw as expressing itself as a kind of dualism—in the sense that resistance requires a force against it. This, then, is the nature of experience itself. Peirce writes:

Whenever we come to know a fact, it is by its resisting us. A man may walk down Wall Street debating within himself the existence of an external world; but if in his brown study he jostles up against somebody who angrily draws off and knocks him down, the sceptic is unlikely to carry his scepticism so far as to doubt whether anything beside the ego was concerned in that phenomenon. The resistance shows him that something independent of him is there. When anything strikes upon the senses, the mind’s train of thought is always interrupted; for if it were not, nothing would distinguish the new observation from a fancy. (CP 1.431)

Now, if the only way we can become aware of existents is when they strike us, this raises the question of their representation. How can a sign, or representation, come to stand for—and convey—the object of an experience or observation, an individual existent? The answer would seem simple enough: it must itself reproduce, or come to embody in its very functioning, the dualism of experience. That is to say, it must itself be dyadically affected by, or connected to, its experienceable object in such a way that awareness of the sign can then point in the direction of it or else lead its interpreter to it. “The index,” writes Peirce, “represents its object by virtue of a character which it could not have if its object did not exist” (CP 5.73). It is a sign that is determined by the existence of its object—and it is in this sense that photography is usually thought of as indexical. However, pointing fingers, the index rerum and nominum of a book, proper names, pronouns are also indices.

In the above example from To Have and Have Not, two sets of proper names were used as indices. We noticed, however, that one set references fictional or imaginary beings. How can we claim these to be existents such that they can be referred to by indices? This is where the notion of a “universe of discourse” (a term Peirce borrowed from Augustus De Morgan—though he also came to use the term “sheet of assertion”) becomes useful. A universe of discourse contains all the objects of a given domain of reference that are taken to be observable and on the basis of which assertions can be made concerning them by using indices. Peirce therefore recognized that fictional beings can be referenced by an index, even though, unlike what happens with non-fictional entities, it is impossible to experience or observe them in any other way than by imagining them through the fiction wherein they appear: “When the universe of discourse relates to a common experience, but this experience is of something imaginary, as when we discuss the world of Shakespeare’s creation in the play of Hamlet, we find individual distinction existing so far as the work of imagination has carried it, while beyond that point there is vagueness and generality” (CP 4.172).6

Once the universe of discourse is established, it is important to see the gap that fiction can introduce in the overlapping of worlds mentioned above. For as soon as one says: “Harry Morgan and Marie Browning are kissing in a hotel in Port de France” or “Harry Morgan takes Johnson on fishing trip on the high seas,” these statements cease to be true of the visual contents of Hawk’s film when we replace the name “Harry Morgan” with “Humphrey Bogart” (the hotel is a movie set in Hollywood7 and rear projections are used for the fishing scenes whenever we see the actors). This should not be a surprise since, after all, the film qua fiction is about Harry Morgan, not Humphrey Bogart. We could try to counter the objection by saying instead: “Humphrey Bogart and Lauren Bacall are kissing while pretending they are in a hotel in Port de France.” Even though this may well be the case, the image itself gives no evidence of this: it merely shows us the couple embracing in a room whose location is fictionally (but not visually) indexed (in good measure through dialogue) as being in a hotel in the capital of Martinique.

This is not to say, however, that the film’s ability to reference individual existents from our world—as opposed to fictional ones—should be neglected. When we appreciate Bogart’s acting it is clearly because the film—not the fictional narrative—records it and thus makes it possible for us to study it by our electing to see this (i.e., individual, particular) performance as one of the objects represented indexically on the screen (though iconicity is also involved through which we apprehend the qualities this particular performance embodies), and whose independent concrete existence we see as affecting the image. It might be argued that as long as the Academy Awards will continue to present actors with Oscars for their performances, indexicality will continue to be a stake in the film experience!8

But the fact remains that part of the experience of watching live-action films (including narrative fiction) is, in fact, the possibility of semiotically using film images for any of the purposes an index may serve, by being existentially affected by its real world object (in this instance: what is recorded by the camera); and that it has been so ever since early cinema, whether or not we engage in it at every moment of watching a film and in full awareness of doing so (for we also engage with the film as an iconic and a symbolic sign, thus serving a host of other semiotic purposes). Indeed, this happens every time the independent existence of the real world captured by the film draws our attention, and makes us turn toward it or attend to it: when in a crime movie I suddenly recognize a street I lived on, or in a melodrama the Paris hotel I once stayed in, or when I search for one of Hitchcock’s cameo appearances, or else when I read the end credits to see what locations were used because I wish to visit the landscapes which I found so arresting while watching the film.9

It is true, of course, that the current omnipresence of visual effects is having a toll on the viewer’s ability to do this accurately and confidently. We may find a landscape especially striking, only to discover it wasn’t really photographed and that none of it actually exists in our world. For although most films continue to shoot material, it is becoming increasingly difficult—impossible at times—for viewers to differentiate what has been recorded by a camera from what is shown without having been filmed, and to distinguish what has been composited together into a shot to create the impression of a unified contiguous space from several heterogeneous elements (both photographic and non-photographic), or what has been digitally manipulated (how many of us saw the facial replacement of Armie Hammer on someone else’s body at a first screening of The Social Network [Fincher, 2010]?)—at least, as long as what is seen doesn’t appear out right as a special effect for the viewer (as is the case with Brad Pitt’s facial replacement in The Curious Case of Benjamin Button). In short, viewers may know that existents from our world are still there to be seen in films, but they’re not quite sure where exactly anymore—since not even the actor’s body is immune to compositing. Does this mean we have no cause to discuss indexicality anymore? In answering, let us begin by taking on the issue from a slightly different perspective and consider this time the filmmaker’s task.

2.

Indices, as I mentioned earlier, are used to represent individual existents. They are the only type of sign that can do this since pure icons merely stand for self-identical qualities, and symbols stand for types (on the ground of laws or habits), neither of which can be said to exist and therefore directly act upon and determine the sign in representing them except as being utterly indeterminate, that is to say, vague (e.g., representing some such possible object) or general (e.g., representing a type of object). The implication, therefore, is that without indices, filmmakers cannot, under any consideration, represent individual existents. Now this might not be entirely a hindrance for fiction filmmaking should we wish to rule out indices standing for individuals that belong to our real-world universe of discourse (as opposed to the fictional world of discourse). For the sake of intelligibility we could let the characters in such a fiction continue to use indices in exchanging amongst themselves (e.g., “come here,” “where is Peter?,” etc.), without it affecting our “no index” rule. The simplest way to achieve this might be to imagine forgoing recording anything with a camera, and therefore to make an animated film that avoids visually representing any individual existent. A good example of this, among others, might be Chuck Jones’s 1942 Bugs Bunny short Case of the Missing Hare, where the friendly cottontail battles it out with a pompous magician in some non-descript, vague space, made up of some exterior decor with a hollow tree (where Bugs resides) and, later, the stage of a theatre (where the magician performs and suffers Bugs’s wrath). Now compare the film to another Bugs Bunny cartoon, French Rarebit (McKimson, 1950), where our lapin friend finds himself in Paris as two chefs seek to turn him into a rabbit stew—sacrebleu!—to their great detriment, of course! Here, however, a series of landmarks are presented verbally and visually, including an unmistakable view of the Eiffel Tower (Figure 1).

Figure 1. French Rarebit, Robert McKimson (1951)
Figure 1. French Rarebit, Robert McKimson (1951)

Obviously, this is a drawn likeness of the famous Paris landmark—what Peirce would call an iconic sign. But it is more than that as well: it is also an index of it—namely, the representation of an individual existent (a non-fictional one!). The fact that it is drawn rather than photographed simply implies that it is not a photographic index, but to be sure it is an indexical sign just the same and one that refers to our universe of real-life existents, even though it is integrated into a fiction. And so too, it follows, is the digitally created Times Square circa 1922 of Baz Lurham’s The Great Gatsby (2013) (Figure 2). To see through this more clearly, we now need to return to Peirce and properly distinguish between icons and indices.

Figure 2. The Great Gatsby, Baz Lurham’s (2013)
Figure 2. The Great Gatsby, Baz Lurham’s (2013)

A pure icon is a self-representing sign in that its connection to its object is one of complete qualitative sameness or self-identity: “A pure icon,” writes Peirce, “does not draw any distinction between itself and its object. It represents whatever it may represent, and whatever it is like, it in so far is. It is an affair of suchness only” (EP2:163). And elsewhere he writes: “No pure Icons represent anything but Forms; no pure Forms are represented by anything but Icons” (CP 4.544). By “form” what is meant here is a quality quite regardless of its hic et nunc manifestation or embodiment in some existent. In other words, it is a mere possibility and, as such, cannot give any indication at all regarding the existence of its object. The upshot is that a pure icon is its object, for the latter is the quality that it itself is as a qualitative essence or continuum (differentiating them as “sign” and “object” fulfills merely logical and functional requirements, not existential or numerical distinctions). The pure icon has no means of signifying its object other than its qualitative self-identity with it. As indicated earlier, only indices can denote particulars—objects that are existentially and thus numerically distinct from the signs that stand for them—by virtue of an existential tie to them, whereas icons have no such purchase on particulars. A pure icon, then, is a very abstract, almost ethereal, kind of representation: it has no concrete existence proper and no “being” other than that of being a mere (self-signifying) possibility, or, to put it differently, it is a possible (as opposed to some actual, embodied) sign of its own self as a possibility. The pure icon is still a sign, but it is so in potentia, not in actu, for only embodied signs act concretely as signs. But what of actual images like paintings, drawings, even photographs and films which we usually think of as iconic, because they represent their object on the ground of some likeness they share with them? Surely these are existents in their own right and as such can be materially distinguished from what they represent. The answer lies in the distinction Peirce makes between pure icons and iconic signs, that is to say, embodied signs that can iconically (i.e., qualitatively, on the ground of a likeness) represent something numerically other than themselves as a possible (as opposed to an existing) object.

When Peirce himself or commentators of his work seek to explain iconicity by offering concrete examples of icons—a figurative painting, a diagram, a map, a sample paint chip, that is, actual artifacts that signify on the ground of a likeness—what is referred to aren’t pure icons, but iconic signs or what Peirce calls hypoicons. Whereas the pure icon is a mere possibility standing for itself as a possibility, a figurative painting is an actually existing thing that while also standing for itself (for its own sui generis qualities as a painting) may as well stand for other formally identical, though numerically distinct, potential manifestations of the form it embodies. (It is worth repeating that iconicity, on its own, cannot offer any assurance as to the existence of its object, whereby the latter is always represented as a mere qualitative possibility, or a form.) In Peirce’s phenomenological language this means that a hypoicon injects an element of Secondness (viz., the duality or otherness of concrete existence) into the semiotic process of iconicity. What this implies is that hypoicons can both self-represent and alio-represent (that is, represent something other than themselves).

Let’s make this more tangible. Imagine an unmistakable (or highly mimetic) painting of a horse. On one hand, the work may be used to signify the qualities it embodies in its depiction of the animal, though quite regardless of the embodiment itself on canvas. This means that the painting can self-signify, or auto-represent itself, not as an existent (which, of course, it is) but as a type of thing of which it is a pictorial embodiment. This could include the form exhibited by the painted horse, but it can also comprise the style of the work. For instance, the art teacher might say to her students: “Just look at the shape of this horse!” to (indexically) bring to their attention and make them appreciate the quality of design-type being embodied on the canvas (for this shape is something general which could also be reprised any number of times). She could also say: “This is so typically Remington,” in which case she is again using the painting as qualitatively self-signifying a type, namely the painter’s style. In both cases the painting is used to self-signify, namely to stand for qualities it possesses and embodies (but doing so regardless of this embodiment as a matter of fact). On the other hand, however, it is also possible to look at the painting as iconically signifying something numerically other than itself, an object with which it nonetheless shares a quality. This is part of what distinguishes the hypoicon from the pure icon. From the perspective of the quality that they share, this other (possible) object—a possible horse—is also a (possible, not an actual) replica or token of a type that the painting embodies, although the painting’s concrete embodiment of it (through paint, brush strokes, lines, etc.) isn’t shared by the object—otherwise this possible token-object would not be numerically distinct from the painting itself. This is a complicated way of saying that now the painting can be seen as iconically representing a (possible) horse, even though an actual painting and a possible horse are two very different sorts of things. In this instance, the hypoicon is alio-representing, since it stands for something other than itself.

If the painting does not represent an actual horse—and only a possible one—it is because nothing about it points toward an existent horse. It is not an index and the horse it depicts could be either an imaginary one, or an actual one: the painting offers no information to this effect. Now, the visual depiction of a merely possible horse is also the depiction of a possible horse-type. That is to say, it may be a black horse or a grey horse, a big or a small horse, a stallion or a mare, or else an Arabian, an Appaloosa or a halter-type Quarter horse, or else it could be one of, say, Tom’s horses or one of the neighbor’s horses, et cetera.10 This means that without an index, the object of our painting is a vague generality (what Peirce calls a negative generality), representing some possible horse which belongs to some possible horse-type. Yet, the possible horse of the painting is less vague than that of the following sentence: “tomorrow I will buy myself a horse” (what will it look like? Will it be male or female? Big or small?), and less positively general than that of the following proposition: “The horse is a noble animal,” by which is meant the entire species and therefore all the individual occurrences of Equus ferus caballus.11 The reason for this is that the painted horse may have color, spots, a given shape, and so on. In other words, it possesses certain qualities that both curtail the realm of possibilities of various aspects of the horse-type (even though any number of such horses may be embodied, e.g., in other paintings, in reproductions and copies, in real-life, etc.), and enable one to refer to it indexically (even though it is not a real horse that one can mount, it can obviously be pointed to: see how, for instance, I can refer to the “saddled brown horse” of Frederick Remington’s Cow Pony). We might add that our ability to recognize and identify the alio-represented object of a hypoicon in a figurative medium—“this is an X-type”—is relative to other embodiments of its object-type, either as mere possibilities (this is what happens with the depictions of unicorns or dragons) or as real worldly existents (as in the case of our painted horse, when the possibility depicted by the painting is congruent with existents from our world, i.e., with real existing horses that concretely embody the horse-type), all of which form a continuum of qualitative possibilities which may be represented through positive generality once named or conceptualized.

In the end, therefore, what distinguishes the pure icon from the hypoicon is the mode of being of the sign. As Peirce writes:

A possibility alone is an Icon purely by virtue of its quality; and its object can only be a Firstness. But a sign may be iconic, that is, may represent its object mainly by its similarity, no matter what its mode of being. If a substantive be wanted, an iconic representamen [a term Peirce often uses as a synonym for “sign”] may be termed a hypoicon. Any material image, as a painting, is largely conventional in its mode of representation; but in itself, without legend or label it may be called a hypoicon. (EP 2:273)
Thus, where a pure icon is a form that is self-representing, an iconic sign is a concrete, embodied—and therefore impure—form that may be used as a self-representation (regardless of its own concrete embodiment) or to alio-represent a possible object with which it shares this same form. It follows that the alio-representing hypoicon is formally, though not materially, identical with its object. In the following passage, Peirce describes the experience of iconicity:

I call a sign which stands for something merely because it resembles it, an icon. Icons are so completely substituted for their objects as hardly to be distinguished from them. Such are the diagrams of geometry. A diagram, indeed, so far as it has a general signification, is not a pure icon; but in the middle part of our reasonings we forget that abstractness in great measure, and the diagram is for us the very thing. So in contemplating a painting, there is a moment when we lose the consciousness that it is not the thing, the distinction of the real and the copy disappears, and it is for the moment a pure dream—not any particular existence, and yet not general [ML: by which Peirce means not positively general]. At that moment we are contemplating an icon. (CP 3.362; emphasis mine)

Taking in the painting, the interpreting mind sees (feels might be a better term to use) a quality, which is to say that the image shows merely the possibility of “something,” that is, some possible and unknown “X” of such or such a description: “X is like this.” If this “X” is identified as a possible (as opposed to an actual) embodiment of the horse-type because the qualitative “like this” is identified as qualities otherwise known to belong to horses, then this interpreting mind says to itself something like (while considering the painting): “This would be a horse if it were real.” However, if a caption is added that reads: “Trigger, the famous movie horse,” then the situation is transformed and the composite of painting and caption can now be used as an index. What is represented is no longer a possible manifestation of the horse-type, but an actual existing embodiment of it: Roy Roger’s legendary horse. The sign is now such that it can point its user toward the existing object of experience that has affected or determined it, so that she can also experience it (at least theoretically; or through other indices of it, if it is an object from the past). This is how we should understand Peirce’s claim that a proposition “represents an image with a label or pointer on it” (CP 5.543).

Let us now complete our understanding of indices before returning to special effects. The index can point in the direction of its object by being really affected by it, yet it doesn’t actually substitute for the object. In the previous example of painting and caption, the move from hypoicon to index must be explained. To a viewer who doesn’t know of the horse called “Trigger,” the painting along with its caption merely says: “something called Trigger is like this,” without pointing to the universe of real-world, observable existents (indeed, the same could be done with the image and caption of a unicorn or a dragon). What is required for this sign to function indexically, therefore, is a certain amount of collateral knowledge concerning its object: one must independently know of Trigger. The surest way to achieve this collateral knowledge is through observation or experience, though in some cases—especially when this is impossible—other signs determined by the object, other indices, may help approximate this observation. In a manuscript from 1907 Peirce writes:

[A] sign is something which functions triadically. [...] Every sign, in functioning as such, produces a mental effect. How shall we name the entire mental effect which a sign by itself is calculated, in its proper significative function, to produce? The word signification is somewhat too narrow, since, as examples will soon show, this mental effect may [also] be of the nature of an emotion or of that an effort. [...] Permit me to call this total proper effect of the sign taken by itself the interpretant of the sign. But merely producing a mental effect is not sufficient to constitute the object of a sign; for a thunderclap or an avalanche may do that without conveying any meaning at all. In order that a thing may be a true sign its proper significate mental effect must be conveyed from another object which the sign is concerned in indicating and which is by this conveyance the ultimate cause of the mental effect. In order to be the cause of an effect,—or efficient cause, as the old phrase was,—it must either be an existent thing or an actual event. Now such things are only known by observation. It cannot be itself any part of the mental effect, and therefore can only be known by collateral observation of the context or circumstances of utterance, or putting forth, of the sign. But the sign may describe the kind of observation that is appropriate and even indicate how the right object is to be recognized. But although the full realization of the meaning requires the actual observation, direct or indirect, of the object, yet a close approach to this may be made by imagining the observation. If the sign is not a true, but only a fictitious sign, it is the mere semblance of a sign. If, however, it be so true as to profess to be in certain respects fictitious, the conditions of a true sign hold in slackened force (EP2:429).

The main idea of this passage is that a sign is a mediation between an object and some consequent effect of that object being the case, what Peirce calls the interpretant. This is what distinguishes the avalanche in itself from a written account or image of it bringing to our mind the idea and horror of the avalanche. (Of course, the avalanche itself may also be a sign of something else, say, global warming, in which case it mediates between an object and an interpretant of its own—although this is a different matter.) But which avalanche? An actual existing avalanche can only be known by experiencing or observing it hic et nunc. However, a sign may describe or indicate the kind of collateral observation to be made in order to realize its meaning12 or else it can point toward the individual existent represented so that it might be observed. As we know, the only way a sign can point in the direction of a particular being or event is through an index. In canonical cases, an index works by referencing a given object set within a universe of discourse which corresponds to the aggregate of individuals which exist (or are experienceable) for the users of the sign. In the above quote, Peirce is explaining that although no index can, by itself, substitute for the role of collateral observation in making known an independently existing individual object, indices may be able to approximate this by combining with an icon (or several of them) so as to help us imagine such observation. This is especially important in cases where collateral observation has become impossible (for instance, if the object lies in the past or isn’t otherwise available for observation: how else can we know of Napoleon today barring opening up his tomb? Moreover any such observation would be somewhat limited in terms of the knowledge it would yield of the man!). Of course, what we thusly imagine—and think to be true (unlike fiction which doesn’t share this pretense)—may turn out to be false or erroneous should no such object actually exist (or have ever existed), in which case says Peirce, we have the mere semblance of an indexical sign. (Think of circumstantial evidence “pointing” the detective in the wrong direction and leading him to falsely accuse someone; or else fake “clues” left by a cunning thief at the scene of a crime which, when “properly” interpreted by the detective, will lead to the wrongful arrest of an innocent person.)

As for fiction, as we saw earlier, Peirce recognized that fictional or imaginary beings may be indexed. Thus, a painting of a man, with the caption “the famous Sherlock Holmes, as he appears in A Study in Scarlet by Arthur Conan Doyle” would be a true indexical sign—one professing its fictionality—though it would be so only in a “slackened way” because it is impossible to experience its object in any other way that to imagine it in relation to Doyle’s fictional account. In fact, we might think of fiction as offering a situation not entirely unlike what is the case with the historical past, with the important caveat that with history there is always the possibility of unearthing new evidence to help us approximate collateral observation made through imagination.

There is a final distinction we need to make about indices. It concerns the fact that not every index points to its object in quite the same way or, better yet, relies on quite the same kind of collateral knowledge in so doing. On this ground, Peirce saw the importance of distinguishing between two classes of indices which he called respectively reagents and designations. He writes:

An index represents an object by virtue of its connection with it. It makes no difference whether the connection is natural, or artificial, or merely mental. There is, however, an important distinction between two classes of indices. Namely, some merely stand for things or individual quasi-things with which the interpreting mind is already acquainted, while others may be used to ascertain facts. Of the former class, which may be termed designations, personal, demonstrative, and relative pronouns, proper names, the letters attached to a geometrical figure, and the ordinary letters of algebra are examples. They act to force the attention to the thing intended. Designations are absolutely indispensable both to communication and to thought. No assertion has any meaning unless there is some designation to show whether the universe of reality or what universe of fiction is referred to. The other class of indices may be called reagents. Thus water placed in a vessel with a shaving of camphor thrown upon it will show whether the vessel is clean or not. If I say that I live two and a half miles from Milford, I mean that a rigid bar that would just reach from one line to another upon a certain bar in Westminster, might be successively laid down on the road from my house to Milford, 13200 times, and so laid down on my reader’s road would give him a knowledge of the distance between my house and Milford. Thus, the expression “two miles and a half” is, not exactly a reagent, but a description of a reagent. A scream for help is not only intended to force upon the mind the knowledge that help is wanted, but also to force the will to accord it. It is, therefore, a reagent used rhetorically. Just as a designation can denote nothing unless the interpreting mind is already acquainted with the thing it denotes, so a reagent can indicate nothing unless the mind is already acquainted with its connection with the phenomenon it indicates. (CP 8.368n23)

If follows, then, that whenever true, undoctored photographs are used indexically to stand for what they depict, they are clearly reagents. Their semiotic potential in this regard lies in the fact that we know how they are produced or, in Peirce’s words, that we are “acquainted with [their] connection with the phenomenon [they] indicate.” In short, there’s no need here for sign users to have prior knowledge of the object signified, since independent knowledge of how the sign reacts to it is sufficient for it to function indexically.13 Equally important for us, however, is the fact that Peirce’s comment suggests that representations such as drawings and paintings can also be truly indexical—even without a caption—as long as we are “already acquainted with the thing [denoted],” which is to say, on the basis of our knowledge that the object thusly drawn to our attention is a particular existent.

3.

And this brings us back to the Eiffel Tower of French Rarebit (Figure 1) or, for that matter, Times Square or the Queensboro Bridge in The Great Gatsby (Figure 2). In both films, these well-known landmarks are represented as specific individual existents, not as vague or general objects. The fact that neither are photographic—at least in the sense of a true, undoctored photograph—does not alter their indexical status. Furthermore, we need to understand that referring to an existent doesn’t imply that the sign will necessarily offer an accurate representation of it in all of its particular qualities (no alio-representing hypoicon sign ever does). Indices only portray qualities when they are accompanied by iconic signs whose function it is to depict by way of exemplification. In a case such as French Rarebit, a rough sketching of the Parisian landmark is sufficient for us to recognize it (in the film, to be sure, our recognition is sustained by other indices as well, including text, dialogue, the recognition of other landmarks, etc.). The same is true for drawn caricatures of well-known people or politicians: they are indexical and iconic signs whose iconicity doesn’t pretend to offer lifelike renderings of their object. (Let us not forget that as far as iconicity is concerned the object is always but a mere possibility, not an existent.) In other words, a sign may be a true index without the iconic sign accompanying it being a “complete” or “true” (in the sense of totally faithful) likeness of its object, which is why, as I mentioned earlier, indexicality and “realism” should be treated separately. Of course, iconicity can play an important role in designations (especially in the case of designation by images), since activating our knowledge of the existence of the object may rely on our ability to recognize it through its likeness.

Yet, even in such cases, the functions of iconicity and indexicality remain distinct. Consequently, liberties taken in drawing the Eiffel Tower or in depicting New York circa 1922 do not impede their function as indices as long as we can recognize them in their haecceity (for instance, there are a couple of websites devoted to historical inaccuracies in the depiction of New York and its buildings in The Great Gatsby—these efforts merely prove that the film is indexical in this respect. Yet it would make no sense to do this with the realm of Dormammu in Dr. Strange! (Figure 3).

Figure 3. The realm of Dormammu in Doctor Strange, Scott Derrickson (2016)
Figure 3. The realm of Dormammu in Doctor Strange, Scott Derrickson (2016)

Photographs are indices that involve, for the most part, fairly accurate likenesses of their real-world objects. As indices, however, they may also, in certain circumstances, combine the two classes of index outlined above. A photograph of a famous landmark will stand for its object both on the basis of our independent acquaintance with this existent and on the ground of our knowledge of the workings of photography. In most situations, however, this doubling of the semiotic function goes easily unnoticed, merely offering redundancy in the way we use the photograph to represent. It might only become obvious when what we otherwise know to be a photograph of something or someone we are acquainted with is so dark or else so out of focus that we can’t make out the object, in which case our ability to use it as a designation based on our recognition of its likeness is jeopardized; or else, when we learn that what we thought was a photographic record is only a semblance of a reagent while still being a designation, a situation certain imperceptible special effects puts us in.

To make all of this more concrete, let’s consider traditional matte paintings. A distinction can be made between imperceptible and invisible mattes on the basis of their congruence with existents from our world. As hypoicons, the mattes of the White House in Dave (Reitman, 1993), of Hindley Hall Mansion in The Paradine Case (Hitchcock, 1947), and of the Death Star in Star Wars (Lucas, 1977) represent their objects along a spectrum of possibilities whereby only the Death Star stands as the pictorial embodiment (all hypoicons are embodied forms) of a possible type of object not congruent with existents of our real-world universe of discourse (i.e., not otherwise embodied in this universe, and therefore not actively a type determining its occurrences, but only a possible type to which the Death Star, as a possible object, would belong if it were real). As a result, matte paintings of the Death Star (Figure 4)—though they show an object which can be indexically referenced in the fiction—are not themselves indices of an actual existing space station, nor can they pass for one (at least for those of us who know that the Death Star isn’t part of the furniture of our real world universe of discourse) even though they may look “real” to the film’s viewers thanks to the photo-realism of the paintings and the way cinematographers smoothly integrate them into the movie.

Figure 4. Photograph of Matte painting for the Death Star by Ralph McQuarrie, Star Wars, George Lucas (1977)
Figure 4. Photograph of matte painting for the Death Star by Ralph McQuarrie, Star Wars, George Lucas (1977)

The situation is different for the other two sets of mattes. Hindley Hall’s interiors pictorially embody a type that is congruent with existents from our universe of discourse: they manifest the same architectural type as do existing eighteenth- or nineteenth-century British country mansions (Figure 5). Thus, if the effect goes undetected,14 the matte paintings are mistaken for photographic indices pointing toward an existent manor which the unsuspecting viewer believes was experienced by the camera and is (or was once) available for collateral observation—which is the raison d’être of the effect, after all (i.e., to be believable in the way photographs are). For the viewer, then, the imperceptible effect is, to use Peirce’s term, the semblance of an index (in this case, a reagent), which is precisely where the deceit lies. This is why we can claim that the deception created by the matte effects in the case of Hindley Hall is of a different nature than that of the Death Star. For although both work hard at deceiving perception, the difference lies in their “embodiment” of an object-type, that is, whether it is a possible object-type—although the viewers might well think it is an actual photographic embodiment—about which we know that replicas exist (some such things as Hindley Hall do exist) or merely a possible object-type which, viewers know, is without any existing replicas (no such things as the Death Star exist, except in the fictional world of Star Wars)—a distinction which belongs to a sort of ontological gap in representation.

Figure 5. Matte painting by Spencer Bagdatopoulos (top) and shot from The Paradine Case, Alfred Hitchcock (1947) (bottom)


Figure 5. Matte painting by Spencer Bagdatopoulos (top) and shot from The Paradine Case, Alfred Hitchcock (1947) (bottom)

The situation of Dave is different, however. Here most of the views of the White House use matte paintings as set extensions. In what follows, I’ll refer only to the view from the North Portico as the First Couple return from an evening “out on the town.” It is a brief shot where the entire building is shown (Figure 6).

Figure 6. Shot of the White House, North Portico in Dave, Ivan Reitman (1993)
Figure 6. Shot of the White House, North Portico in Dave, Ivan Reitman (1993)

Technically speaking, what is going on here is fairly simple: a glass painting is used to stand-in for a “normal” shot taken by a camera. In semiotic terms, the filmmakers are using the semblance of a reagent to imperceptibly imitate a photographic view of the White House. However, because the matte painting shows a likeness of a well-known landmark which we recognize, it offers viewers a real index, namely, a designation. If we look at this more closely, we will see that it is in the relation between these two classes of indices that the deception of the special effect lies.

We know that what viewers are actually looking at, although most aren’t aware of this, is a photographic image of a painting of the White House (the shot that we see in the film is a composite, but let’s leave this out until later). Now, a condition for the success of the effect is that the photographic depiction of the painting be such that its object—that is, the White House as it is made immediately available by way of a pictorial representation of it—comes to substitute itself for the actual object of the photographic image that appears on the film screen. In other words, viewers must not be made aware that they are looking at the photographic depiction of a painting. Rather, they must see through these layers to “access” the White House. Put simply, the effect requires of the photographic image that it act as nothing more than a “dumb,” transparent window. It appropriates the object of the painting as its own, even though in so doing the viewer takes the shot as depicting the real White House instead of depicting a depiction of it by way of a painting. What viewers are looking at and what they see or believe they are looking at are therefore two distinct matters. Because the glass matte is quite imperceptible and photo-realistic, viewers who don’t know that using the grounds of the White House to shoot a narrative film is forbidden (except perhaps for its occupants!) may thus be inclined to believe that they are really seeing the building—that is, really seeing a photographic image of it—rather than a painted image of it used as trompe l’œil.15

We also know that a painted likeness of the White House can be used as an index of it for those who recognize the building. It turns out that matte artist Paul Lasaine worked from photographs of the White House to get the likeness of the building right with those photos standing for the real building as both index and hypoicon of it. Yet however much photographs may be affected by, and resemble, what they depict, they also possess their own sui generis qualities, namely through lens distortion, stock grain, or in how they react to light, et cetera, and a good matte artist will take this into consideration. The matte painter’s task, in other words, is to replicate how an object looks when filmed, not so much how it looks to the naked eye. The north entrance of the White House in Dave is shown with some slight lens tilt distortion and light glare that reproduce how the building might look when photographed at night. Whereas such qualities of the image can normally be used as indices of the functioning of a camera (and not of the existent object depicted by the image!), because this is a painting, they are in this case as much a part of what is being depicted as the White House itself. For Lasaine and the filmmakers, the pictorial rendering of glare and lens distortion are used as hypoicons (they embody a type of “apparatus effect” through likeness), not indices—and certainly not as indices of the photographic nature of the image.16 For the unsuspecting viewer fooled by the trompe l’œil, however, the lens effect and glare are merely a semblance of an indexical sign of the apparatus—and thus part of the overall deception (cf. Ball 2017:S171–72). As for the image of the White House, while it is clearly indexical, it is only so—though unbeknownst to the viewer—on the basis of object recognition and thus as a designation. It is therefore unrelated to photography’s connection to visible existents making up our real universe of discourse: the index, in other words, isn’t where the viewer thinks it is!

What is interesting to note here is how, regardless of whether or not Lasaine copied an actual photograph, the indexical status of the painting literally crosses over to the film for the unsuspecting viewers who recognize the building. For it carries over not because the film image is a photographic index of Lasaine’s painting of the White House (perhaps this is what Lasaine sees when watching the film!), but because it is an iconic sign of it, which ipso facto makes it an iconic sign of the White House, as well as an index of it. We can put it this way: as a photographic record of Lasaine’s painting, the film image is framed in such a way as to capture and reproduce (one is tempted to say, appropriate) some of its visual qualities, including the likeness of the White House which, for viewers with minimal worldly knowledge, is immediately recognized. On the basis of this recognition the image can be used as an index of its object, thus further determining what is otherwise (erroneously) believed to be a photographic index of the famous landmark.

The upshot is that the view of the North Portico is a designation; but the deceit is that, for viewers convinced by the matte painting into believing they are seeing a photographic view of the building, the reagent that superadds itself to the designation is merely a semblance. And because the trickery concerns the way the object appears—how it is represented—and not its actual existence, the only way to discover the deception is to gain knowledge about the production of the sign (i.e., find out how the shot was made), since no actual collateral observation of the object (the White House) would reveal it to be a semblance of a reagent (i.e., a “faked” photographic image).

Compare what is going on in Dave to the discussion regarding Bogart and Bacall at the outset of this paper. Again, two universes of reference are superimposed: the White House as the official residence and workplace of the real-life President of the United States and the White House as the residence and workplace of the film’s fictional President.17 Both buildings, the existent and the fictional one, can be referred to indexically with true propositions as long as their universes of discourse are carefully distinguished. In a sense, perhaps, this would be a bit more like Bogart “playing” himself in a fictional film. But there is another, deeper, difference, for those two universes are not superimposed in exactly the same fashion. In To Have and Have Not, there is a way of describing all that which we see the characters doing as being necessarily true of both the characters and the actors: for instance, although actor Walter Sande doesn’t die when the character of Johnson is killed during a shootout, the visible bodily “performances” of actor and character are both identical and existentially tied—with those of the latter being determined by the former—in such a way that it is true to claim that both the actor and the character can be seen lying on the floor in their respective worlds. The overlap can be explained by the fact that, as long as one plays along with the convention of cinematic fiction, a number of visible qualities that belong to the actor are transferred to the character through their hypoiconic embodiment on the screen. Moreover, the photographic nature of the process implies that the image, which offers a likeness of the character-embodying actor, also serves as a trace of him. In Dave, on the other hand, the image of the White House, while it is a bona fide index pointing toward its object, is not a photographic index of the building (although this is not perceptually apprehended by the viewer). As we can see, it isn’t that indexicality has disappeared, but one class of index (reagent) has been replaced by another (designation) and the photographic layering of iconicity and indexicality has been disturbed. As a result, fiction can assert its rightful course, which is precisely what takes place when the film partially merges the two universes of discourse (the real and the fictional) in a composite shot that shows an actual car driving up to the building. The composite result, therefore, is really a complex hypoicon that is a likeness typifying a situation, but one that involves an indexical element referencing our world—for instance, an actual/fictional car of such and such a make driving up to the White House some evening (the actual evening can only be referenced in the fictional world). Yet it is a hypoicon that seeks to pass for a photographic index, for a reagent. And it does so by using in turn a designative index (Lasaine’s painting of the White House which is photographed and rendered indexically, though invisibly so for the unwary viewer), the semblance of a reagent (the photo-realism of the painting and its hypoiconic rendering of photographic qualities), a true photographic reagent (the moving car, which also involves a photographic hypoicon) all for the purpose of tricking the viewer into believing that what is shown is a single contiguous space belonging to both universes of discourse (reality and fiction), when it only belongs—as far as the complete composite is concerned—to that of fiction (no such actual car really drove up to the White House on the day the scene was shot).

When successfully fulfilling its intended purpose, an “effect shot” such as the matte composite of Dave is not a special effect for the viewer (who does not see or register it), though it is one for the filmmakers who manipulate the image in postproduction. The difference is especially striking for effects we call “invisible” as they involve this time the viewer’s disbelief at the semblance of indexicality (with regards to the real world and to CG-simulated photographic images of it) that special effect films offer.18 Depending on the type of effect, its photo-realism and its ability to blend with the truly indexical aspect of the film, this disbelief collides with what is perceived and with how we have come to habitually interpret photographic images (and what today may pass for photographic images).

The blowing-up of the White House in Independence Day (Emmerich, 1996)—which used a miniature whose real blowing-up was recorded—is a case in point. No reasonable viewer believes this to be a photographic record of the actual White House being destroyed in order to make the film, even if this is what it looks like. Here, what needs to be considered are the conditions that explain why a statement such as “This is the White House” can stand as a true description of the image in both the real and the fictional universes of discourse, while “This is the White House exploding” may only do so with regards to fiction, and “This is a miniature of the White House (exploding)” is suppressed even though it is not entirely so (which is why the effect is not imperceptible, although it is invisible). It is here wherein resides what could be called the perceptual fiction of (perceptible) special effects, as the ontological gap calls on a fiction based on indexing our real-world universe of discourse as designations perceptually impersonate reagents (Independence Day’s image of a miniature—itself a reagent—posing as an image of the real White House) or else on using pictorial embodiments of object-types that are merely congruent with existents from our universe of discourse yet—and herein lies the special effect—in a way that breaks the practical, financial/technical, or ontological constraints that otherwise restrict the use of a camera in recording a situation or event. (For one of many examples, consider the destruction of any purely fictional building as, for instance, Tony Stark’s mansion in Iron Man 3 [Black, 2013].)

As we can see, a film’s ability to refer to our world of existents is not limited to its use of photographic means. We could add that when Albert Dieudonné (Napoléon [Gance, 1927]) or Marlon Brando (Désirée [Blaustein, 1954]) play Napoléon, they are also used by the filmmakers as indices of the well-known empereur des Français—though not through photographic means. The same is true of any reconstructed worldly event: from the storming of the Winter Palace in October (Eisenstein, 1928) to D-Day landing on Omaha Beach in Saving Private Ryan (Spielberg, 1998). Such indices are more like names, designations, than reagents. In each case, the particulars they signify are integrated to a fiction, yet the dual or hybrid status of what they stand for—indexed reality (presented with the help of icons) and iconic fiction (indexed with regards to the universe of discourse of the fiction)—is part of the experience they offer viewers.

Although the above examples of matte shots and miniature far from cover the entire range of visual and special effects (imperceptible and invisible), it seems clear that filmmakers’ aspirations to fashion images that break free from, modify or divert the camera’s attachment to the real, cine-photographable world, still relies in most cases on the idea of a camera’s indexical potential in order to create perceptual fictions and forms of deceit whose experience19 and meaningfulness require this idea. Whether or not they actually involve the representation of an existent from our world with which spectators are already acquainted, depicting possible occurrences of object-types consistent with our world, or else showing impossible objects or events, effects typically offer semblances of photographic reagents—feeding off our acquaintance of the photographic process—whenever they mimic the texture and qualities of camera-produced images, that is to say, whenever they offer viewers an icon of live-action cinema. Effects in this instance—from Méliès to CGI—may be seen to exist in the gap that forms between the idea of a camera’s indexical potential and efforts to curtail its deployment, all the while exploiting it.

Visual and special effects want it both ways: they seek to free the cinema from the camera’s close dependence on reality, and yet they also aspire to give viewers the impression of being realized through the recording action of a camera—even when it is obvious that this cannot be the case. What is especially interesting about this fact is that, in doing so, visual and special effects help us become aware of how difficult it actually is to come to terms with the role or function of photographic indexicality in cinema, including in what I’ve referred to as “zero degree” fiction filmmaking.

Concluding Remarks

As intimated earlier, it seems obvious that without photographic reagency, the cinema would not have developed its star system. Characters and stars, however, are merely the most visible manifestations of the separate tracks that meet in the image: reality and fiction. Indeed, whenever cinema is used according to its baseline, we should think of the real world of particulars captured by the camera as an underlying layer always ready to pierce or tear through the fabric(ation) of the fiction as soon as viewers—for whatever reason or purpose—“retrieve” their awareness that in looking at the film they are also looking at a representation of the real world and its particulars. Hence the idea that the film’s photographic images may be used indexically. On the other hand, it is also clear that filmmakers—especially in classical American and European cinema—make efforts in the opposite direction by offering viewers the depiction of fictional worlds. In the celluloid era, most images in live-action fiction films were recorded with a camera (and notwithstanding a few exceptions, visual and special effects occupied but a marginal amount of screen time in any given film).

But what exactly were filmmakers seeking to represent in the process? Were they seeking to represent the particulars of our world or were they instead trying to create fictional worlds filled with their own particulars (except for overlaps like the White House, Times Square or the Eiffel Tower) by way of qualities made manifest in our world? As long as films relied mostly on photographic images the question perhaps was merely “academic”: reality and fiction cohabited in both Hollywood musicals and in Italian Neo-Realist films. The difference in these films was stylistic and artistic, but it had nothing to do with photographic indexicality. Neo-Realist films, of course, sought to represent reality, the lives of Italians in the aftermath of WWII. The films sometimes have an almost documentary feel to them, and yet they are still fictions whose characters, situations, environment are meant to stand, for the viewer, not for actual existents but for exemplary types of existents. The reality indexed by the camera can thus fade next to that which the fiction constructs—what is at stake here is realism, not photographically indexed reality (even if both are very close). Though the viewer may use images of Roma città aperta (Rossellini, 1945) or Ladri di biciclette (De Sica, 1948) indexically, shooting mostly on location, using the real world as its material, also gives the images in these films a flavor, a quality of worldly presence relative to those locations such as only photography can provide, and perhaps it is this quality that was sought out by a Rossellini or a De Sica, rather than the representation of each real-world particular recorded by the camera (need I emphasize that I think this to be the case?). The fact is, however, that this quality—this icon—accompanies all instances of photographic indexicality. Wherein lies what I referred to as the paradox of using images of real particulars to create fictional worlds. In this regard alone (I’m not discussing here aspects of mise-en-scène or editing, which, as we know, have an important role to play in the stylistic identity of Neo-Realism), the key difference between an MGM musical and a Neo-Realist film lies in choosing what is being recorded.

Now, if visual and special effects throw this paradox into relief it is because they interfere with the way fiction cinema meshes the two tracks of reality and fiction as the camera knots together indexicality and iconicity. Indeed, unlike what I have called “zero degree” cinema, special effects’ perceptual fictions can more readily appear as offering a model of cinematic “semioticity” where iconicity dominates over photographic indexicality, one where we have hypoiconicity (blended indexicality and iconism) without pure indexicality; which is to say, a semioticity content with designations rather than reageants. In both cases, needless to say, iconism and indexicality are at issue, but in different ways relative to the myth of photographic indexicality as referred to by Bazin and others in discussing realism in cinematic fiction.

This becomes especially obvious when we consider motion capture (MOCAP). It has been argued that MOCAP is indexical since the technology for it requires recording the actual actions and movements of an individual object or body.20 But the fact of the matter is that MOCAP makes it very difficult (though not entirely impossible, mind you) to use the end result seen on screen in such a way as to point toward that individual existent in its haecceity, except in a very vague fashion.21 Given this difficulty, might it not be a more accurate description of MOCAP to see it as extracting general qualities from an individual existent and transferring them to an inexistent fictional being? In which case MOCAP would “iconize” rather than “index”: it’s concern would lie not in representing the actual fact of qualitative embodiment in an existent, but in representing the qualities an existent manifests regardless of its existence by transferring them to the representation of an imaginary being. Perceived from this angle, MOCAP can be seen to loosen the knot that otherwise binds indexicality and iconicity in photography by separating qualities from an existent, even if the end effect is to offer a semblance of index by mimicking the feel of motion photography as much as possible. In this regard, MOCAP can be used to remind us of how much live-action fiction cinema—from Neo-Realism to Avatar—thrives on likeness, more so perhaps than on the representation of individual existents, even though it can’t seem to entirely do away with them. Could it be then that “iconizing”—the transferring (the stealing?) of qualities from the real world over to the world of fiction—has always been the goal of live-action fiction cinema, even when filmmakers use their camera to stick as close as possible to worldly existents so as to capture as many aspects of reality as possible? Is photographic indexicality, notwithstanding what I wrote above regarding the star system and the different ways it can manifest itself in the experience of viewing, merely, then, an epiphenomenon in the way we use fiction film?

These questions aren’t meant to argue that all fiction cinema since L’arroseur arrosé (Lumière, 1895)—perhaps the first fiction film made with a camera—neatly folds into visual and special effects, and has always been indifferent to reality. After all, since the whole point of much visual and special effects, especially today, lies in their ability to masquerade as photographic and therefore build onto photography’s association with the index and its representation of individual existents, the perceptual fiction they set up seems to ask of us that we entertain the notion (either as consciously held false/fictional belief, as in the case of Star Wars; or as mistaken belief in the case of Dave) that what we are seeing was directly present to the camera in the way that we now perceive and come to understand it on the screen. What this points to in the end is the need for untangling a number of questions concerning cinema and our use(s) of it, and the fact that our thinking about film must take into consideration the entire domain of visual and special effects.

Acknowledgments

I wish to thank Constantine Nakassis and Meghanne Barker for their fine editorial comments.

Endnotes

1. For the purposes of this paper I use the terms visual effects and special effects interchangeably, though in a longer work devoted to these issues I offer reasons why both terms are necessary for our critical metalanguage (Lefebvre and Furstenau forthcoming).

2. These constraints can be grouped in the following way: (a) sundry practical constraints, such as legal, ethical or security constraints: one shouldn’t commit a crime to illustrate the beheading of Mary Stuart; shooting in the vicinity of a real tornado or storm may prove too hazardous; (b) financial and technical constraints: it often costs less to use mattes than to build an entire building a set, and it is often technically more convenient in terms of “image control” to shoot the live action on the studio lot and add mattes later than to shoot a scene on location; (c) “ontological” constraints: Star Wars’ Death Star only exists in fiction, as do the characters of King Kong (Cooper and Shoedsack, 1933) and The Invisible Man (Whale, 1933), and have no real-world equivalents. Therefore, they cannot be captured by the camera without some form of deceit. Moreover, ontological constraints also can affect the film apparatus: no real existing camera can move in the way the “viewpoint” is seen to move in the famous CG-enhanced long take in Panic Room (Fincher, 2002) as burglars break into a house.

3. Except, of course, for those that are medium-specific: framing, black and white, absence of sound, absence of “real” depth, et cetera.

4. In itself, of course, this phenomenon is nothing new: theatre also juxtaposes real and fictional worlds; only, theater does so without the mediation of a recording device.

5. Of course, there are many hybrid cases where a fictional world employs as its characters, events or settings, people, events and places that have (or have had) a corresponding real-world existence.

6. The same argument holds for the indexing signs of mathematics or logic whereby they can point to demonstrations or to elements internal to the “world” of a problem. In Peirce’s scheme there is no need to distinguish between “external” reference and “internal” (or co-textual) reference.

7. The real-world location of this room is nowhere indicated in the film, of course. However, anyone investigating the whereabouts of Humphrey Bogart at the time of shooting his scenes would discover that he was at Warner Brothers Studios’ Stage 28 in Burbank, California.

8. This is merely a way of emphasizing that without photographic indexicality it is difficult to imagine how cinema could have developed the star system that has been so vital for its success. Here too, however, things are changing as actor’s performances may be enhanced and transformed through digital alterations with the body/face split of facial replacement or the altering of facial features or movements—for instance, in Dark Shadows (Burton, 2012), Johnny Depp’s blinking eyelids were eliminated from his face.

9. Of course, these are not the only indices I can use in my experiencing a film. There’s an indefinite number of them which may point to their objects in as many directions. For example, looking at the image I can see it as an index that a wide angle or a telephoto lens is being used, or that the cameraperson is operating a Steadicam. Certain stylistic features may also point to the filmmakers who use them, much like a signature points to its owner. However, such indices are related to the apparatus and its use, not to the photographic nature of the image as imprint of the profilmic world that lies in front of the camera. Choosing one index over the other is relative to one’s purpose as an interpreter of signs. Note that in all those cases where the index points to what is seen as the occurrence of a type or a habit (as may be the case with a filmmaker’s style), what is really at stake is an instantiation, token or replica of what Peirce calls a symbol, that is, a general type related to its object—this object being itself of the nature of a type—on the grounds of a rule, a convention, or a habit (i.e., a general principle capable of an indefinite number of occurrences, which is what an artistic style is). In such a case, the sign and the object as well as the interpretation of their relation are all tokens of a type—individual instantiations of something abstract and general—as is the singular, existential relation that obtains between the sign and its object.

10. It could also depict a horse type that has no real-world token: a child’s drawing of a pink horse, for instance.

11. In CP 1.427, Peirce distinguishes negative and positive generality in the following terms : “Generality is either of that negative sort which belongs to the merely potential, as such, and this is peculiar to the category of quality; or it is of that positive kind which belongs to conditional necessity, and this is peculiar to the category of law.”

12. In a different section of the same manuscript (which contains substantially different drafts of an unpublished article), Peirce writes:

It should be mentioned that though a sign cannot express its Object, it may describe, or otherwise indicate, the kind of collateral observation by which that Object is to be found. Thus, a proposition whose subject is distributively universal (not plural or otherwise collectively universal) such as “Any man will die,” allows the interpreter, after collateral observation has disclosed what single universe is meant, to take any individual of that universe as the Object of the proposition, giving, in the above example, the equivalent “If you take any individual you please of the universe of existent things, and if that individual is a man, it will die.” If the proposition had been, “Some Old Testament character was translated,” the indication would have been that the individual must be suitably selected; while the interpreter would have been left to his own devices to identify the individual (EP2:408).
Notice that in such cases Peirce is discussing signs (symbols, actually) whose objects are initially represented as either general or vague and require additional indexical determination to fulfill their meaning. This section of the manuscript was published, with an introduction by Helmut Pape, under the title “Charles S. Peirce on Objects of Thought and Representation” (Pape 1990).

13. This has long been noted by scholars of photography and cinema. Jean-Marie Schaeffer called such collateral knowledge the arché of photography, namely the fact that “a photograph functions as an indexical image only as long as we know that it is a photography and know what this fact implies” (Schaeffer 1987:42; my translation). The same idea is expressed by Philip Rosen when he writes: “this capacity [to see film as indexical] must include a knowledge about how the signifier [sic] was supposed to be produced” (Rosen 2001:21).

14. In this case, discerning eyes might notice the eerie atmospheric effect—soft focus and grain characteristic of the era’s matte shots using an optical printer—in the long shots of the manor’s rooms, which used matte paintings, distinguishing them from the closer, sharp focus shots made on sets with props. Some inconsistent floor shadows can also be noticed as the lawyer Keane (Gregory Peck) is about to exit Mrs. Paradine’s room. Given the diegetic source of light, his shadow should extend forward in the foreground where other shadows appear on he painted matte—not to mention some continuity issues: a piano seen on set in a close shot seems to be absent from the matte painting used for long shots of Mrs. Paradine’s room.

15. Of course, someone might also think the building is a set, a facade, built on a studio lot—which is how it might have been done prior to the use of glass shots or Schüfftan process shots. However, I surmise most people who know this much about how films are made would quickly dismiss the idea on the basis of cost in relation to how briefly the North Portico appears in the film.

16. Of course, indexicality could also be involved: should these “photographic” effects be determined by an actual photograph embodying those same qualities, anyone aware of this relation (such as the production team) could theoretically use the matte painting as an indexical sign pointing to this photograph.

17. The fact that the film’s titular character, Dave Kovic, is merely a stand-in for the comatose and dying elected President—in the spirit of Chaplin’s The Great Dictator (1940)—could be seen as a fun thematic duplication (though most likely by fluke, since this is an age-old theme of narrative fiction and theatre) of what is going on perceptually with the matte shots: deception based on likeness.

18. Here, and in what follows, I refer to “invisible” and “imperceptible” effects following a distinction made many years ago by Christian Metz (1977). According to him, invisible effects, which comprise effects like those of The Invisible Man (Whale, 1933), are effects that do not appear to affect the film image’s ability to represent the world and yet are nonetheless perceptible in the sense that the viewer clearly realizes or else “senses” that some form of deceit or mystification—whose means may well be left undetermined—is happening. Imperceptible effects, on the other hand, are those forms of trickery that go entirely undetected by the viewer.

19. For viewers, of course, the experience of imperceptible effects is basically a non-experience.

20. See, for instance, Grossoli 2011.

21. One would first have to recognize that what is shown is produced through motion capture and know independently that the technology requires the recording of a body’s actual movements and actions. The same, of course, is true of photography. However, by not providing us with a likeness with which to identify the individual whose movements are recorded and shown in the image, motion capture also keeps us in the dark as to its identity. Perhaps like a photograph of a person whose face and body are covered by shadows, we see a vague existing something whose qualities in this case lie only in the forms of the movements captured. In certain instances, of course, the actual existent can still be identified, especially when facial movements are captured: one can get recognizable glimpses of Andrew Serkis in installments of the Planet of the Apes series or of Zoe Saldana in Avatar (Cameron, 2009).

References

Allison, Tanine. 2016. The Modern Entertainment Market Place, 2000–Present: Special/Visual Effects. In Editing and Special/Visual Effects, edited by C. Keil and K. Whistle, eds. New Brunswick, NJ: Rutgers University Press.

Ball, Christopher. 2017. Realisms and Indexicalities of Photographic Propositions. Signs and Society 5(S1):S154–77.

Bazin, André. 2009. The Ontology of the Photographic Image. In What is Cinema?, translated by T. Barnard, 3–12. Montreal: Caboose.

Grossoli, Marco. 2011. Motion/Performance Capture and the Afterlife of the Index: A Reconsideration of André Bazin’s “Myth of Total Cinema.” Recherches Sémiotiques/Semiotic Inquiry 31(1–3):152–73.

Gunning, Tom. 2007. Moving Away from the Index: Cinema and the Impression of Reality. differences 18(1):29–52.

Lefebvre, Martin and Marc Fursteneaeu, eds. Forthcoming. Faking the View: Special Effects on Screen from Méliès to Motion Capture. Amsterdam: Amsterdam University Press.

Morgan, Daniel. 2006. Rethinking Bazin: Ontology and Realist Aesthetics. Critical Inquiry 32(3):443–81.

Metz, Christian. 1977. Trucage and the Film. Critical Inquiry 3(4):657–75.

Pape, Helmut. 1990. Charles S. Peirce on Objects of Thought and Representation. Noûs 24(3): 375–95.

Peirce, Charles Sanders. 1931–58. The Collected Papers of Charles S. Peirce. 8 vols., edited by C. Hartshorne, P. Weiss, and A. W. Burks. Cambridge, MA: Harvard University Press. [cited as CP]

Peirce, Charles Sanders. 1998. Essential Peirce, Volume 2: Selected Philosophical Writings, 1893–1913, edited by Peirce Edition Project. Bloomington: Indiana University Press. [cited as EP2]

Rosen, Philip. 2001. Change Mummified: Cinema, Historicity, Theory. Minneapolis: University of Minnesota Press.

Schaeffer, Jean-Marie. 1987. L’Image Précaire: Du Dispositif Photographique. Paris: Seuil.