Video: Digital Seriality: Code & Community in the Super Mario Modding Scene

Above you’ll find the video of my talk, “Digital Seriality: Code & Community in the Super Mario Modding Scene,” which I delivered on September 27, 2016 as part of the Interactive Media & Games Seminar Series at Stanford University.

Here is the abstract for my talk:

Digital Seriality: Code & Community in the Super Mario Modding Scene

Shane Denson

Seriality is a common feature of game franchises, with their various sequels, spin-offs, and other forms of continuation; such serialization informs social processes of community-building among fans, while it also takes place at much lower levels in the repetition and variation that characterizes a series of game levels, for example, or in the modularized and recycled code of game engines. This presentation considers how tools and methods of digital humanities — including “distant reading” and visualization techniques — can shed light on serialization processes in digital games and gaming communities. The vibrant “modding” scene that has arisen around the classic Nintendo game Super Mario Bros. (1985) serves as a case study. Automated “reading” techniques allow us to survey a large collection of fan-based game modifications, while visualization software helps to bridge the gap between code and community, revealing otherwise invisible connections and patterns of seriality.

Playing the Photographer: Creative Self-Expression through In-Game Photography — Jason Lajoie

The Last of Us™ Remastered_20140801163128

Jason Lajoie’s abstract for the panel “Generativity and Creative Agency in Post-Cinematic Media” at SLSA 2016 in Atlanta:

Playing the Photographer: Creative Self-Expression through In-Game Photography

Jason Lajoie, University of Waterloo

The photo editor mode offered in The Last of Us Remastered (TLOUR), Naughty Dog’s 2014 Playstation 4 port of their 2013 Playstation 3 game, offers players the means to take a photo at any point in the game. This option alters and can also elevate user engagement within the game. It can also make statements about self-expression through gameplay, as notably illustrated by conflict photographer Ashley Gilbertson, who applied the same techniques he acquired in real-world conflict zones to the fictitious battlegrounds in the game, achieving his photographs by considering his situatedness as real spectator in virtual environments. From a game studies perspective, the photo editor mode enables new ways for players and designers to think about game design, and offers innovative means of expression for players to interact in a creative game space. My investigation draws on Roland Barthes’s exploration into the affective capacity of photographs, and Jose Van Dijck’s claim that the malleability and manipulability of digital photography affects the formation of identity by repurposing our memories and means of communication. What are the affective resonances of photographs on real spectators when the spectrum itself is virtual? By exploring the use of photo editor modes in TLOUR, and in other Playstation 4 titles like The Order: 1886, I consider the ways this program expands the affordances of gameplay and narrativity by providing players interactive means for creative expression in otherwise restrictive and linear game modes.

On the ‘Parergodic’ Work of Seriality in Interactive Digital Environments

 

batman_and_parergodicity-slides-JPG.002

Here is the full text of my talk from the 2015 conference of the Society for Literature, Science, and the Arts — part of the panel on Video Games’ Extra-Ludic Echoes (organized by David Rambo, and featuring talks by Patrick LeMieux and Stephanie Boluk, David Rambo, and myself).

On the “Parergodic” Work of Seriality in Interactive Digital Environments

Shane Denson (SLSA 2015, Houston, Nov. 15, 2015)

I want to suggest that popular serialized figures function as indexes of historical and media-technical changes, helping us to assess the material and cultural transformations that such figures chart in the process of their serial unfoldings. This function becomes especially pronounced when serial figures move between and among various media. By shifting from a medial “inside” to an “outside” or an in-between, serial figures come to function as higher-order media, turning first-order apparatic media like film and television inside out and exposing them as reversible frames. But with the rise of interactive, networked, and convergent digital media environments, this outside space is called into question, and the medial logic of serial figures is transformed in significant ways. This transformation, I suggest, is not unrelated to the blurring of relations between work and play, between paid labor and the incidental work culled from our entertainment practices. In the age of transmedia, serial figures move flexibly between media much like we move between projects and contexts of consumption and production. The dynamics of border-crossing that characterized earlier serial figures have now been re-functionalized in accordance with the ergodic work of navigating computational networks—in accordance, that is, with work and network forms that frame all aspects of contemporary life.

I will come back to this argument in a moment and elaborate with reference to Batman and his movement from comics to film and video games, but first let me say a few words about “the work of seriality.” In the nineteenth century, production work became increasingly serialized as it was fragmented and mechanized in factories, culminating in the assembly line, the paradigmatic site of serialized production, and eventually leading to digital automation and process control. Ultimately, of course, this trajectory was spearheaded by capitalism’s own seriality, or its structuration as an endless series of M-C-M’ progressions. At the same time, works of culture also fell under the spell of the series. The industrial steam press churned out penny dreadfuls and dime novels, before comic strips, film serials, radio and TV series took over. The simultaneous rise of serialized work practices and serial “works” of culture is too massive, I suggest, to be a sheer coincidence. And concomitant with these two was a third form of serialization: that of cultural identity or of subjective experience itself. As Benedict Anderson and Jean-Paul Sartre before him have argued, new forms of community, identity, and perception were based in the serial work of media, such that, for example, the serialization of daily papers, consumed more or less simultaneously by an entire nation, could produce the nation itself as an “imagined community” of serialized subjects. Anderson’s conception of the serialities of nationhood or the proletariat suggests a material connection between the minute level of concrete serial media practices and the broad level of discursive, cultural, or imagined realities—a connection that I want to pursue into the realm of digital media.

batman_and_parergodicity-slides-JPG.004

I want to suggest that serialized media are able to leverage these shifts in the nature of work/works because they function according to the logic of the “parergon,” as described by Jacques Derrida. Etymologically, the term parergon is composed of the prefix “para-” (next to, or beside) and “ergon,” which derives from Greek for work. The parergon is thus literally “next to the work,” marginal or supplementary to it, as a frame is with respect to a painting (or an hors d’oeuvre with respect to the main course of a meal).

parergon

But the picture frame in particular demonstrates an essential reversibility: on the one hand, the frame serves as a background for the work, as a ground for the image it frames and selects or presents. On the other hand, the frame can also be absorbed into the figure when seen against the larger background of the wall, as when we take a broad view of a row of paintings on a museum wall before selecting one to observe more closely. The frame is therefore subject to repeated figure/ground reversals, and it’s the same with serialized media, which are constituted in the flickering interplay between an ongoing sequence and its articulation into discrete segments.

batman_and_parergodicity-slides-JPG.017

A serial figure like Frankenstein’s monster embodies this interplay and mediates it as a higher-order reflection on media change. The monster is of course part of a film’s diegetic universe, for example, but it also exceeds that frame and partakes in a plurimedial series of instantiations. We never just see Frankenstein’s monster; we see an iteration of the monster that stands in extradiegetic relation to Karloff’s iconic portrayal and to a series of media and mediations of the figure. And we should not forget that Karloff’s mute monster, which contrasts sharply with the eloquent monster of Shelley’s novel, once served to foreground the transition from silent cinema to the talkies. Figure/ground reversibility is an essential precondition for plurimedial seriality as such, specifically enabling the foregrounding of mediality that allows the serial figure to serve as a figuration of media change.

But with the rise of digital media, the formerly discrete media across which serial figures were deployed come to mingle in much closer proximity. What Henry Jenkins calls our “convergence culture” responds by coming up with new ways to tell stories (and to sell commodities) that take advantage of the coming-together of media in the space of the digital. In Jenkins’s version, transmedia storytelling is inherently serial, but much less linear than a conventional television series might be, as it allows the reader/viewer/player/user to explore various facets of a story-world through movies, games, textual and other forms, allowing for a variable order of consumption that corresponds, we might say, to the database structures in which digital information is stored and (interactively) accessed. But transmedia storytelling often aims to smooth over the disjunctures between media installments; the parergonal logic of figure/ground reversals that sustained serial figures and allowed them to track and foreground media changes is thus transformed. A serial figure like Batman thrives in this new environment and traces this transformation in relation to computational mediation and the shift from a parergonal to what I call a parergodic logic.

batman_and_parergodicity-slides-JPG.020

With the term parergodics, I want to link Derrida’s notion of the parergon with Espen Aarseth’s use of the term ergodics to describe processes and structures of digital interactivity. Ergodics combines the Greek ergon (work) and hodos (path), thus positing nontrivial labor as the aesthetic mode of players’ engagement with games. For Aarseth, the arduous or laborious path of ergodic interactivity marks a fundamental difference between digital media such as video games or electronic literature on the one hand and traditional literature and narrative media on the other. For whereas the path of a narrative is fixed for the reader of a novel or the spectator of a film, it must be generated in digital media through a cooperative effort between the user and the computational system. The signs composing the text of a video game—including textual strings, visual perspectives, narrative and audiovisual events—are not (completely) predetermined but generated on the fly, in real time, as the player makes his or her way through the game. Ergodics, the path of the work or the work of the path, therefore describes the nontrivial labor at the heart of gameplay. But to expand this beyond Aarseth’s narrower frame of reference, the concept of ergodics can also be seen to ground a wider variety of interactive and participatory potentials in contemporary culture, where computational networks are implicated virtually ubiquitously in entertainment, social life, and work. The borders between these realms are remarkably unclear (think of all the things people do on social networks and the virtual impossibility of distinguishing clearly between work activities and play), and it would seem that this has something to do with the indifference of computational media to the type of contents processed. This computational indifference to the phenomenological modalities of human experience – or to the differences between the analogue media that at least partly corresponded to those modalities – leads, as Mark Hansen argues, to a divergence between mediation in its classical, perceptually oriented form and a new form of mediation that channels human affect into the process-oriented project of establishing ever greater networks of pure connectivity. This is the larger significance, I propose, of what Steven Shaviro calls “post-cinematic affect”: in contrast to the cinema, which was constituted by the storage and reproduction of perceptual objects, ergodic mediation involves acts of affective interfacing with the fundamentally post-perceptual realm of computation, which is algorithmic, distributed, and nonlocal, in contrast to the phenomenological basis of human embodiment. Clicking on a Youtube video not only delivers perceptual content to your embodied eyes and ears, it also delivers computational content – information about affective, epistemic, and monetary valuations – to the routines of network-constitutive algorithms. In this environment, play activities not only involve the execution of nontrivial work, as Aarseth argues, but corporations and financial interests, among others, continually find clever ways to disguise work as play, to “gamify” our labor, both paid and unpaid, while mining the data generated in the process in order to profit from both dedicated and incidental work. In this environment, as Matteo Pasquinelli has argued, virtually any investment of attention or affect will also generate a surplus value for Google, Facebook, etc. – a value produced and accumulated parasitically, without regard for any significance we may attach to the contents of our digital interactions, by means of computational algorithms functioning on an altogether different level than the human concerns that feed them.

As a result, media “contents” become incidental or marginal to work, so that our so-called “participatory culture” might better be termed a “parergodic culture,” where cultural “contents” are reversibly supplemental to the nontrivial labor of interactive work. But the notion of “parergodic culture” suggests also that there might be para-ergodic margins from which to witness the shift, to take stock of it in the process of its occurrence. This is where the parergon meets ergodics, and it’s in this reversible margin of parergodicity, neither completely inside nor outside the realm of ergodics, that I’d like to situate the serial work of Batman from about the mid 1980s to the present.

batman_and_parergodicity-slides-JPG.021

The starting point is the appearance of graphic novels such as The Dark Knight Returns (Frank Miller, 1986), Arkham Asylum: Serious House on Serious Earth (Grant Morrison, 1989), and The Killing Joke (Alan Moore, 1988), which re-envisioned Batman as a darker figure and laid the groundwork for the figure’s medial self-awareness.

In their wake, a key scene in Tim Burton’s 1989 film stages a parergonal reversal of medial spaces during Joker’s televised address to Batman and the people of Gotham. The medium takes on an unexpected materiality as the Joker shoves the mayor’s image off the screen, and a crucial reversal is visualized as a shot of several contiguous studio monitors gives way to the various screens united in Batman’s multimedia console. It is here, with a sudden freeze frame interaction, that Batman enacts a further parergonal reversal: while the film’s editing leads us to believe that Bruce Wayne, like all the citizens of Gotham, is viewing the Joker’s address live, he pauses the recording, in effect pausing the continuity of the film itself. And with this seemingly insignificant difference it introduces between live and recorded images, Batman’s pausing of the image announces, in effect, an entry into the interactive space of post-cinematic media. This is the first step towards the reconceptualization of images and visual media as purely processual, computational, and no longer tied to perception as its objects.

batman_and_parergodicity-slides-JPG.024

Jump ahead twenty years. Computational technologies are implemented more broadly in the actual production of visual media, for example in post-cinematic blockbusters like Christopher Nolan’s Batman trilogy. Nolan’s second film, The Dark Knight, can be seen as a serial continuation not only of Batman Begins, the first film in Nolan’s trilogy, but also an updating of Burton’s early exploration of Batman’s ergodic mediality.

batman_and_parergodicity-slides-JPG.027

Most centrally, Nolan’s film updates Batman’s console and places it in the middle of the caped crusader’s pursuits to restore order to Gotham. The film spends a considerable amount of time foregrounding this computational wonder machine, which alternates reversibly with the film itself and serves to foreground its CGI-based spectacles.

parergodic-batman

Within the frame of the narrative, a new range of computational powers is demonstrated, including biometric facial recognition and computational forensics. Early in the film, Bruce Wayne’s tech guy Lucius Fox demonstrates to him a new technology, utilizing a cell phone to emit an inaudibly high frequency capable of mapping a remote location by means of digitally enhanced sonar. This sets the stage for the film’s climax, when the sonar program is spread, virus-like, to the cell phones of all of Gotham’s inhabitants. Through this network, which feeds into Bruce Wayne’s central console, now equipped with a giant wall of display devices, Batman is able to “see” the whole city.

sonar-vision

This is a disembodied or nonlocalized 3D computer-graphics vision generated through a distributed, nonhuman sensory form that substitutes computational process for perceptual object. Seeing through the eyes of a machinic network, Batman is able to find the bad guys just in time for the final showdown, but at a decisive moment Batman’s “vision” machine crashes.

sonar-crash

The event is presented to us in first-person perspective, crucially drawing attention to the mediation of our own vision through computational processes. Here the parergonal reversibility between diegetic and medial levels is thoroughly parergodic, as we are made witness to an event that challenges the perceptual frames delineating the narrative and our ability to engage or disengage with the medium.

batman_and_parergodicity-slides-JPG.030

But the scene anticipates an even more intense experience of parergodic involvement in the video game Arkham Asylum. Here, a specifically parergonal exploration of spatialized boundaries between sanity and insanity that goes back to the graphic novel of the same title is translated into a narrative that weaves back and forth between “reality” and Scarecrow-induced hallucinations. The player, who has to act in order to stay alive, can never be sure when one of these hallucinatory states has begun, and he or she therefore gets drawn into such illusions until an abrupt awakening takes place in the wake of a victory (in a boss battle) or its deferral. Even more poignantly, though, there is a total break with all narrative, perceptual, and actional involvement at one point late in the game, when the images on the screen freeze and display digital artifacts and the soundtrack begins to skip like a scratched CD.

Suddenly, the screen goes black and the game literally reboots – at least, I could swear that my PlayStation restarted at this moment, while a feeling of panic gripped me. When the game restarts, we see images reminiscent of the game’s opening scenes – thus compounding a sense of fear that either my disk or my machine is broken, and that all my progress in the game, by this time some 10 or 20 hours, is lost and will have to be repeated from the beginning. But this time things are backwards: the Joker’s in the driver’s seat, escorting Batman into Arkham Asylum. The cutscene gives way to an interactive sequence where the player controls the Joker, thus instituting a weird sort of actional identification with the villain, who then turns and points a gun directly at the player, whose vision is suddenly realigned with the perspective of Batman. There’s no chance to avoid death, and we see this “Mission failed” screen with the tip, “Use the middle stick to avoid Joker’s gun fire.” Only, there is no middle stick on the PlayStation or Xbox controller. This whole sequence therefore emphasizes the point of interface as a reversible margin where computational or ergodic media converge as both the thematic/actional “content” and the material platform for play.

batman_and_parergodicity-slides-JPG.035

And the quasi-glitch and simulated crash of the game channel this attention to reveal the significant work involved in ergodic play—the very real panic and extradiegetic fears activated here highlight the cognitive and physical labor invested by the player, the precariousness of the digital platform for the storage or accumulation of such work, over which we have little individual control, though our activities are sure to generate profit for the corporations holding ownership of intellectual properties (like Batman), of proprietary software and hardware (like the console we’re operating), or the algorithms that will mine our activities for surplus value. This, I suggest, is parergodic culture.

batman_and_parergodicity-slides-JPG.036

Video Games’ Extra-Ludic Echoes — SLSA 2015

machine-puzzled-them

I am excited to be a part of the panel “Video Games’ Extra-Ludic Echoes,” which will be chaired by my colleague David Rambo at the upcoming conference of the Society for Literature, Science, and the Arts (SLSA) — hosted in Houston this year by Rice University, November 12-15, 2015. Below you will find the panel description and links to the individual abstracts.

Video Games’ Extra-Ludic Echoes

Chair: David Rambo

Each of the three presentations in this panel perform their own media-theoretical approach to comprehend how video games extend and consolidate the sociotechnical logics surrounding their conception, production, and reception. We intend to kindle a discussion about the ways certain video games order significant ideologies and activities of human life: from multimedial culture industries to the blurred division between life and labor to the concealment of racism in the techniques of 20th-century entertainment. All three share a motivation to delineate various cultural and economic inheritances of video games and the transformative ways in which video games echo those inheritances.

Shane Denson’s contribution attends to the serial manifestations of Batman across genres and media. He hones in on the subsumption of life-time under work-time that is common to the computational networks of daily life and to the crossed borders of serial figures such as Batman. In Stephanie Boluk and Patrick LeMieux’s “White Hand, Black Box,” we learn to recognize in the maniculed pointer of various softwares’ interfaces and especially in the Master Hand of Super Smash Brothers the now concealed tradition of black minstrelsy as it was imbued by Disney in the gloved hands of Mickey Mouse and other characters. David Rambo deploys a Spinozist theoretical framework to categorize the video game, Diablo III in particular, as a neoliberal enterprise that extrinsically determines the player’s desire and even will to live. Spinoza’s Ethics thereby offers a way to conceptualize the video game both as an autonomous entity marked with finite, and thus fulfillable, completeness, and as a node in a much broader regime of affections that orders capital’s socioeconomic system.

These three presentations depict the video game as an artifactual conduit eminently bound up with the cultural forces that ineluctably structure our civilization, from its marginal groups to its most powerful systemic imperatives.

Abstracts for the individual papers:

Shane Denson, “Gaming and the ‘Parergodic’ Work of Seriality in Interactive Digital Environments”

Stephanie Boluk and Patrick LeMieux, “White Hand, Black Box: The Manicule from Mickey to Mario to Mac OS”

David Rambo, “Spinoza on Completion and Authorial Forces in Video Games”

Gaming and the ‘Parergodic’ Work of Seriality in Interactive Digital Environments — Shane Denson

arkham-hack

My abstract for the panel “Video Games’ Extra-Ludic Echoes” at SLSA 2015 in Houston:

“Gaming and the ‘Parergodic’ Work of Seriality in Interactive Digital Environments”

Shane Denson, Duke University and Leibniz University of Hannover

Twentieth-century serial figures like Tarzan, Frankenstein’s monster, or Sherlock Holmes enacted a “parergonal” logic; as plurimedial figures, they continually crossed the boundaries between print, film, radio, and televisual media, slipped in and out of their frames, and showed them – in accordance with a Derridean logic of the parergon – to be reversible. In the twenty-first century, the medial logics of serial figures have been transformed in conjunction with the rise of interactive, networked, and convergent digital media environments. A figure like Batman exemplifies this shift as the transition from a broadly “parergonal” to a specifically “parergodic” logic. The latter term builds upon Espen Aarseth’s notion of “ergodic” gameplay – where ergodics combines the Greek ergon (work) and hodos (path), thus positing nontrivial labor as the aesthetic mode of players’ engagement with games. These new, ergodic serial forms and functions, as embodied by a figure like Batman, raise questions about the blurring of relations between work and play, between paid labor and the incidental work culled from our entertainment practices. Following Batman’s transitions from comics to graphic novels, to the films of Tim Burton and Christopher Nolan, and on to the popular and critically acclaimed Arkham series of videogames, I will demonstrate that the dynamics of border-crossing which characterized earlier serial figures has now been re-functionalized in accordance with the ergodic work of navigating computational networks – in accordance, that is, with work and network forms that frame all aspects of contemporary life.

White Hand, Black Box: The Manicule from Mickey to Mario to Mac OS — Stephanie Boluk & Patrick LeMieux

mac-manicule

Stephanie Boluk and Patrick LeMieux’s abstract for the panel “Video Games’ Extra-Ludic Echoes” at SLSA 2015 in Houston:

“White Hand, Black Box: The Manicule from Mickey to Mario to Mac OS”

Stephanie Boluk, UC Davis, and Patrick LeMieux, UC Davis

Whereas the manicule, a pointing finger directing a reader’s attention, has been used for a millennium in chirographic and print texts, in the context of twentieth century animation and twenty-first century computing the medieval pointer has been recontextualized as the hand of the animator to a graphic user interface (GUI) element. After the popularization of the talkie in the late twenties, in Steamboat Willie (1928), the first ever “Merry Melody” released by Disney, Mickey Mouse adopts gloves and the lilting voice of Al Jolson’s Jazz Singer (1927). This process sanitizes a genre of racist comedy for mainstream consumption. Although Mickey’s gloves are easily deemed merely a contrivance of the technical limitations related to articulating fingers in early animation, Bimbo and Betty, Oswald and Ortensia, Foxy and Roxy, and, of course, Mickey and Minnie are anthropomorphic animals that whitewashed their relation to racist caricatures inspired by blackface minstrelsy. This history was further obfuscated as “Mickey’s manicules” eventually found themselves as elements within the contemporary operating systems like Mac OS and as GUI’s within videogames like Mario Paint in the eighties and nineties. From the metaleptic manicule of classic animations to the metonymic manicule in the GUI, this paper ultimately performs a close reading of the figure of “Master Hand” in Super Smash Bros. (1998) in order to argue that the white hand allegorizes the ways in which “user friendly” design has black boxed the racialized history of computation.