Seriality Seriality Seriality

seriality-seriality-seriality

Poster for the upcoming conference “Seriality Seriality Seriality: The Many Lives of the Field that Isn’t One” (final conference of the Popular Seriality Research Unit), taking place June 22-24, 2016 at the Freie Universität Berlin!

More info at the conference website.

Visualizing Digital Seriality, Or: All Your Mods Are Belong to Us!

Visualizing_Digital_Seriality

In this post, I want to outline some ongoing work in progress that I’ve been pursuing as part of my postdoctoral research project on seriality as an aesthetic form and as a process of collectivization in digital games and gaming communities. The larger context, as readers of this blog will know, is a collaborative project I am conducting with Andreas Jahn-Sudmann of the Freie Universität Berlin, titled “Digital Seriality” — which in turn is part of an even larger research network, the DFG Research Unit “Popular Seriality–Aesthetics and Practice.” I’ll touch on this bigger picture here and there as necessary, but I want to concentrate more specifically in the following on some thoughts and research techniques that I’ve been developing in the context of Victoria Szabo’s “Historical & Cultural Visualization” course, which I audited this semester at Duke University. In this hands-on course, we looked at a number of techniques and technologies for conducting digital humanities-type research, including web-based and cartographic research and presentation, augmented and virtual reality, and data-intensive research and visualization. We engaged with a great variety of tools and applications, approaching them experimentally in order to evaluate their particular affordances and limitations with respect to humanities work. My own engagements were guided by the following questions: How might the tools and methods of digital humanities be adapted for my research on seriality in digital games, and to what end? What, more specifically, can visualization techniques add to the study of digital seriality?

I’ll try to offer some answers to these questions in what follows, but let me indicate briefly why I decided to pursue them in the first place. To begin with, seriality challenges methods of single-author and oeuvre or work-centric approaches, as serialization processes unfold across oftentimes long temporal frames and involve collaborative production processes — including not only team-based authorship in industrial contexts but also feedback loops between producers and their audiences, which can exert considerable influence on the ongoing serial development. Moreover, such tendencies are exacerbated with the advent of digital platforms, in which these feedback loops multiply and and accelerate (e.g. in Internet forums established or monitored by serial content producers and, perhaps more significantly, in real-time algorithmic monitoring of serialized consumption on platforms like Netflix), while the contents of serial media are themselves subject to unprecedented degrees of proliferation, reproduction, and remix under conditions of digitalization. Accordingly, an incredible amount of data is generated, so that it is natural to wonder whether any of the methods developed in the digital humanities might help us to approach phenomena of serialization in the digital era. In the context of digital games and game series, the objects of study — both the games themselves and the online channels of communication around which gaming communities form — are digital from the start, but there is such an overwhelming amount of data to sort through that it can be hard to see the forest for the trees. As a result, visualization techniques in particular seem like a promising route to gaining some perspective, or (to mix metaphors a bit) for establishing a first foothold in order to begin climbing what appears an insurmountable mountain of data. Of particular interest here are: 1) “distant reading” techniques (as famously elaborated by Franco Moretti), which might be adapted to the objects of digital games, and 2) tools for network analysis, which might be applied in order to visualize and investigate social formations that emerge around games and game series.

Inter-Ludic-Seriality

Before elaborating on how I have undertaken to employ these approaches, let me say a bit more about the framework of my project and the theoretical perspective on digital seriality that Andreas Jahn-Sudmann and I have developed at greater length in our jointly authored paper “Digital Seriality.” Our starting point for investigating serial forms and processes in games and gaming communities is what we call “inter-ludic seriality” — that is, the serialization processes that take place between games, establishing series such as Super Mario Bros. 1, 2, 3 etc. or Pokemon Red and Blue, Gold and Silver, Ruby and Sapphire, Black and White etc. For the most part, such inter-ludic series are constituted by fairly standard, commercially motivated practices of serialization, expressed in sequels, spin-offs, and the like; accordingly, they are a familiar part of the popular culture that has developed under capitalist modernity since the time of industrialization. Thus, there is lots of continuity with pre-digital seriality, but there are other forms of seriality involved as well.

Intra-Ludic-Seriality

“Intra-ludic seriality” refers to processes of repetition and variation that take place within games themselves, for example in the 8 “worlds” and 32 “levels” of Super Mario Bros. Here, a general framework is basically repeated while varying and in some cases increasingly difficult tasks and obstacles are introduced as Mario searches for the lost princess. Following cues from Umberto Eco and others, this formula of “repetition + variation” is taken here as the formal core of seriality; games can therefore be seen to involve an operational form of seriality that is in many ways more basic than, while often foundational to, the narrative serialization processes that they also display.

Para-Ludic-Seriality

Indeed, this low-level seriality is matched by higher-level processes that encompass but go beyond the realm of narrative — beyond even the games themselves. What we call “para-ludic seriality” involves tie-ins and cross-overs with other media, including the increasingly dominant trend towards transmedia storytelling, aggressive merchandising, and the like. Clearly, this is part of an expanding commercial realm, but it is also the basis for more.

Serial-Superstructure

There is a social superstructure, itself highly serialized, that forms around or atop these serialized media, as fans take to the Internet to discuss (and play) their favorite games. In itself, this type of series-based community-building is nothing new. In fact, it may just be a niche form of a much more general phenomenon that is characteristic for modernity at large. Benedict Anderson and Jean-Paul Sartre before him have described modern forms of collectivity in terms of “seriality,” and they have linked these formations to serialized media consumption and those media’s serial forms — newspapers, novels, photography, and radio have effectively “serialized” community and identity throughout the nineteenth and twentieth centuries.

Infra-Ludic-Seriality

Interestingly, though, in the digital era, this high-level community-building seriality is sometimes folded into an ultra low-level, “infra-ludic” level of seriality — a level that is generally invisible and that takes place at the level of code. (I have discussed this level before, with reference to the BASIC game Super Star Trek, but I have never explicitly identified it as “infra-ludic seriality” before.) This enfolding of community into code, broadly speaking, is what motivates the enterprise of critical code studies, when it is defined (for example, by Mark Marino) as

an approach that applies critical hermeneutics to the interpretation of computer code, program architecture, and documentation within a socio-historical context. CCS holds that lines of code are not value-neutral and can be analyzed using the theoretical approaches applied to other semiotic systems in addition to particular interpretive methods developed particularly for the discussions of programs. Critical Code Studies follows the work of Critical Legal Studies, in that it practitioners apply critical theory to a functional document (legal document or computer program) to explicate meaning in excess of the document’s functionality, critiquing more than merely aesthetics and efficiency. Meaning grows out of the functioning of the code but is not limited to the literal processes the code enacts. Through CCS, practitioners may critique the larger human and computer systems, from the level of the computer to the level of the society in which these code objects circulate and exert influence.

Basically, then, the questions that I am here pursuing are concerned with the possibilities of crossing CCS with DH — and with observing the consequences for a critical investigation of digital game-based seriality. My goal in this undertaking is to find a means of correlating formations in the high-level superstructure with the infra-ludic serialization at the level of code — not only through close readings of individual texts but by way of large collections of data produced by online collectives.

Romhacking

As a case study, I have been looking at ROMhacking.net, a website devoted to the community of hackers and modders of games for (mostly) older platforms and consoles. “Community” is an important notion in the site’s conception of itself and its relation to its users, as evidenced in the site’s “about” page:

ROMhacking.net is the innovative new community site that aggressively aims to bring several different areas of the community together. First, it serves as a successor to, and merges content from, ROMhacking.com and The Whirlpool. Besides being a simple archive site, ROMhacking.net’s purpose is to bring the ROMhacking Community to the next level. We want to put the word ‘community’ back into the ROMhacking community.

The ROMhacking community in recent years has been scattered and stagnant. It is our goal and hope to bring people back together and breathe some new life into the community. We want to encourage new people to join the hobby and make it easier than ever for them to do so.

Among other things, the site includes a vast collection of Super Mario Bros. mods (at the time of writing, 205 different hacks, some of which include several variations). These are fan-based modifications of Nintendo’s iconic game from 1985, which substitute different characters, add new levels, change the game’s graphics, sound, or thematic elements, etc. — hence perpetuating an unofficial serialization process that runs parallel to Nintendo’s own official game series, and forming the basis of communal formations through more or less direct manipulation of computer code (in the form of assembly language, hex code, or mediated through specialized software platforms, including emulators and tools for altering the game). In other words, the social superstructure of serial collectivity gets inscribed directly into the infra-ludic level of code, leaving traces that can be studied for a better understanding of digital seriality.

But how should we study them? Even this relatively small sample is still quite large by the standards of a traditional, close reading-based criticism. What would we be looking for anyway? The various mods are distributed as patches (.ips files) which have to be applied to a ROM file of the original game; the patches are just instruction files indicating how the game’s code is to be modified by the computer. As such, the patch files can be seen, rather abstractly, as crystallizations of the serialization process: if repetition + variation is the formal core of seriality, the patches are the records of pure variation, waiting to be plugged back into the framework of the game (the repeating element). But when we do plug it back in, then what? We can play the game in an emulator, and certainly it would be interesting — but extremely time-consuming — to compare them all in terms of visual appearance, gameplay, and interface. Or we can open the modified game file in a hex editor, in which case we might get lucky and find an interesting trace of the serialization process, such as the following:

2014-12-02 12.23.19 pm

Similar to Super Star Trek with its REM comments documenting its own serial and collective genesis, here we find an embedded infratext in the hexcode of “Millennium Mario,” a mod by an unknown hacker reportedly dating back to January 1, 2000. Note, in particular, the reference to a fellow modder, “toma,” the self-glorifying “1337” comment, and the skewed ASCII art — all signs of a community of serialization operating at a level subterranean to gameplay. But this example also demonstrates the need for a more systematic approach — as well as the obstacles to systematicity, for at stake here is not just code but also the software we use to access it and other “parergodic” elements, including even the display window size or “view” settings of the hex editor:

Millennium-Resize-Infratext

In a sense, this might be seen as a first demonstration of the importance of visualization not only in the communication of results but in the constitution of research objects! In any case, it clearly establishes the need to think carefully about what it is, precisely, that we are studying: serialization is not imprinted clearly and legibly in the code, but is distributed in the interfaces of software and hardware, gameplay and modification, code and community.

Again, I follow Mark Marino’s conception of critical code studies, particularly with respect to his broad understanding of the object of study:

What can be interpreted?

Everything. The code, the documentation, the comments, the structures — all will be open to interpretation. Greater understanding of (and access to) these elements will help critics build complex readings. In “A Box Darkly,” discussed below, Nick Montfort and Michael Mateas counter Cayley’s claim of the necessity for executability, by acknowledging that code can be written for programs that will never be executed. Within CCS, if code is part of the program or a paratext (understood broadly), it contributes to meaning. I would also include interpretations of markup languages and scripts, as extensions of code. Within the code, there will be the actual symbols but also, more broadly, procedures, structures, and gestures. There will be paradigmatic choices made in the construction of the program, methods chosen over others and connotations.

In addition to symbols and characters in the program files themselves, paratextual features will also be important for informed readers. The history of the program, the author, the programming language, the genre, the funding source for the research and development (be it military, industrial, entertainment, or other), all shape meaning, although any one reading might emphasize just a few of these aspects. The goal need not be code analysis for code’s sake, but analyzing code to better understand programs and the networks of other programs and humans they interact with, organize, represent, manipulate, transform, and otherwise engage.

But, especially when we’re dealing with a large set of serialized texts and paratexts, this expansion of code and the attendant proliferation of data exacerbates our methodological problems. How are we to conduct a “critical hermeneutics” of the binary files, their accompanying README files, the ROMhacking website, and its extensive database — all of which contain information relevant to an assessment of the multi-layered processes of digital seriality? It is here, I suggest, that CCS can profit from combination with DH methods.

2014-12-08 03.44.57 pm

The first step in my attempt to do so was to mine data from the ROMhacking website and paratexts distributed with the patches and to create a spreadsheet with relevant metadata (you can download the Excel file here: SMB-Hacks-Dec1). On this basis, I began trying to analyze and visualize the data with Tableau. But while this yielded some basic information that might be relevant for assessing the serial community (e.g. the number of mods produced each year, including upward and downward trends; a list of the top modders in the community; and a look at trends in the types of mods/hacks being produced), the visualizations themselves were not very interesting or informative on their own (click on the image below for an interactive version):

2014-12-08 04.02.56 pm

How could this high-level metadata be coordinated with and brought to bear on the code-level serialization processes that we saw in the hexcode above? In looking for an answer, it became clear that I would have to find a way to collect some data about the code. The mods, themselves basically just “diff” files, could be opened and compared with the “diff” function that powers a lot of DH-based textual analysis (for example, with juxta), but the hexadecimal code that we can access here — and the sheer amount of it in each modded game, which consists of over 42000 bytes — is not particularly conducive to analysis with such tools. Many existing hex editors also include a “diff” analysis, but it occurred to me that it would be more desirable to have a graphical display of differences between the files in order to see the changes at a glance. My thinking here was inspired by hexcompare, a Linux-based visual diff program for quickly visualizing the differences between two binary programs:

hexcompare

However, the comparison here is restricted to local use on a Linux machine, and it only considers two files at a time. If this type of analysis is to be of any use for seriality studies, it will have to assess a much larger set of files and/or automate the comparison process. This is where Eric Monson and Angela Zoss from Visualization & Information Services at Duke University came in and helped me to develop an alternative approach. Eric Monson wrote a script that analyzes the mod patch files and records the basic “diff” information they contain: the address or offset at which they instruct the computer to modify the game file, as well as the number of bytes that they instruct it to write. With this information (also recorded in the Excel file linked to above), a much more useful and interactive visualization can be created with Tableau (click for an interactive version):

2014-12-08 04.38.04 pm

Here, Gannt charts are used to represent the size and location of changes that a given mod makes to the original Mario game; it is possible to see a large number of these mods at a single glance, to filter them by year, by modder, by title, or even size (some mods expand the original code), etc., and in this way we can begin to see patterns emerging. Thus, we bring a sort of “distant reading” to the level of code, combining DH and CCS. (Contrast this approach with Marino’s 2006 call to “make the code the text,” which despite his broad understanding of code and acknowledgement that software/hardware and text/paratext distinctions are non-absolute, was still basically geared towards a conception of CCS that encouraged critical engagements of the “close-reading” type. As I have argued, however, researching seriality in particular requires that we oscillate between big-picture and micro-level analyses, between distant readings of larger trends and developments and detailed comparisons between individual elements or episodes in the serial chain.)

But to complete this approach, we still need to correlate this code-based data with the social level of online modding communities. For this purpose, I used Palladio (a tool explicitly designed for DH work by the Humanities + Design lab at Stanford) to graph networks on the basis of metadata contained in Readme.txt files.

shout-outs

Here, I have mapped the references (“shout-outs,” etc) that modders made to one another in these paratexts, thus revealing a picture of digital seriality as an imagined community of modders.

community-references

Here, on the other hand, I have mapped references from paratextual materials associated with individual mods to various online communities that have come and gone over the years. We see early references to the defunct TEKhacks, by way of Zophar’s Domain, Acmlm’s and Insectduel’s boards, with more recent references to Romhacking.net, the most recent community site and the one that I am studying here.

smb3-references

As an example of how the social network and code-level analyses might be correlated, here I’ve filtered the network graph to show only those modders who refer in their paratexts to Super Mario Bros. 3 (hence bringing inter-ludic seriality to bear on their para- and infra-ludic interventions). The resulting graph reveals a small network of actors whose serializing activity involves mixing and referencing between SMB1 and SMB3, as well as between each other. The Tableau screenshot on the right then selects just these modders and reveals possible similarities and sites of serialization (for closer scrutiny with hexcompare or tools derived from the modding community itself). For example, we find that the modder AP’s SMB3-inspired patches from September 2005 and flamepanther’s SMB DX patches from Oct 2005 exhibit traces of possible overlap that deserve to be looked at in detail. The modder insectduel’s After World 8 (a mod that is referenced by many in the scene) from February 2006 shares large blocks around 31000-32000 bytes with many of the prolific modder Googie’s mods (which themselves seem to exhibit a characteristic signature) from 2004-2006. Of course, recognizing these patterns is just the beginning of inquiry, but at least it is a beginning. From here, we still have to resort to “close reading” techniques and to tools that are not conducive to a broad view; more integrated toolsets remain to be developed. Nevertheless, these methods do seem promising as a way of directing research, showing us where to look in greater depth, and revealing trends and points of contact that would otherwise remain invisible.

Finally, by way of conclusion and to demonstrate what some of this more detailed work looks like, I’d like to return to the “Millennium Mario” mod I considered briefly above. As we saw, there was an interesting infratextual shoutout and some ASCII art in the opening section of the hexcode. With Tableau, we can filter the “diff” view to display only those mods that exhibit changes in the first 500 bytes of code, and to map that section of code in greater resolution (this is done with the slider in the bottom right corner, marked “Start” — referring to the byte count at which a change in the game starts):

Millennium-and-other-earlybytemods1

Here we find two distinctive (visual) matches: viz. between “Millennium Mario” and Raysyde’s “Super Mario Bros. – Remix 2” from 1999, and between ATA’s “Super Mario Bros. – Yoshi’s Quest” and Krillian’s “Mario Adventure 2,” both from 2000. The latter two mods, while clearly different from the former two, also exhibit some overlap in the changes made to the first 20 or so bytes, so it will be interesting to compare them as well.

hexcompare-SMBRemix2-Millennium-b

Now we can use hexcompare for finer analysis — i.e. to determine if the content of the changed addresses is also identical (the visual match only tells us that something has been changed in the same place, not whether the same change has been made there).

hexcompare-SMBRemix2-Millennium

Here we find that Raysyde’s “Super Mario Bros. – Remix 2” does in fact display the same changes in the opening bytes, including the reference to “toma” and the ASCII art. This then is a clear indication of infra-ludic serialization: the borrowing, repetition, and variation of code-level work between members of the modding community. This essentially serial connection (an infra-serial link) would hardly be apparent from the level of the mods’ respective interfaces, though:

2000-01-01-Millennium1999-01-01-SMBRemix2

When we compare “Millennium Mario” with ATA’s “Super Mario Bros. – Yoshi’s Quest,” we find the ASCII art gone, despite the visual match in Tableau’s mapping of their “diff” indications for the opening bytes:

hexcompare-Millennium-YoshisQuest

“Yoshi’s Quest” corresponds in this respect to Krillian’s “Mario Adventure 2”:

hexcompare-YoshisQuest-MarioAdv2

Thus we have another clear indication of infra-ludic serialization, which would hardly have been evident other than by means of a directed filtering of the large dataset, in conjunction with a close analysis of the underlying code.

Again, however, this is just the beginning of the analysis — or more broadly of an encounter between DH and CCS. Ideally, the dataset would be expanded beyond ROMhacking.net’s database; other online communities would be mined for data; and, above all, more integrative tools would be developed for correlating social network graphs and diff-maps, for correlating community and code. Perhaps a crowdsourced approach to some of this would be appropriate; for what it’s worth, and in case anyone is inclined to contribute, my data and the interactive Tableau charts are linked above. But the real work, I suspect, lies in building the right tools for the job, and this will clearly not be an easy task. Alas, like digital seriality itself, this is work in progress, and thus it remains work “to be continued”…

Thanks finally to Eric Monson, Angela Zoss, Victoria Szabo, Patrick LeMieux, Max Symuleski, and the participants in the Fall 2014 “Historical & Cultural Visualization Proseminar 1” at Duke University for the various sorts of help, feedback, and useful tips they offered on this project!

Out Now: Serialization in Popular Culture

serialization in pop culture

Just back from a trip abroad, I was happy to find this in the mailbox: my copy of Serialization in Popular Culture, edited by Rob Allen and Thijs van den Berg. The volume goes back to an excellent conference that took place in Amsterdam in 2011, organized by the editors of the book, where I presented a paper on film serials of the 1910s: “Rethinking the Serial-Queen Melodrama: Serial Narration and Medial Self-Reflexivity in Transitional-Era Cinema.” Now much expanded, my paper appears here as “The Logic of the Line Segment: Continuity and Discontinuity in the Serial-Queen Melodrama.” The book contains a number of wonderful contributions by Mark W. Turner, Joyce Goggin, Dan Hassler-Forest, Sean O’Sullivan, Jason Dittmer, and more:

TOC-Serialization-in-pop-culture-1

TOC-Serialization-in-pop-culture-2

Also, I’ve posted this before, but I can’t resist posting once more my colleague and collaborator Ruth Mayer’s high praise for the volume:

“This collection presents an ambitious and original intervention in the field of seriality studies. It captures the workings of serialization as a core principle of modernity by taking stock of a wide range of medial formats and narrative and non-narrative configurations from the nineteenth century to the present time.” – Ruth Mayer, University of Hanover, Germany

Finally, the book is, unfortunately, quite expensive in the hardcover version that is now available, and it is to be hoped that a paperback edition will appear at some point. In the meantime, if you are in a position to do so, please request a university library or other institution to order a copy, and in this way support the editors and contributors and increase the chances for an affordable paperback/ebook edition.

Digital Seriality

mario-luigi-pixel-gnome-2

Just in time for the holidays, a new issue of the open-access journal Eludamos: Journal for Computer Game Culture is now online. Among the articles in this issue is a piece that I co-authored with Andreas Jahn-Sudmann, called “Digital Seriality: On the Serial Aesthetics and Practice of Digital Games.” In this article we put forward some of the central ideas of our joint research project and provide illustrations of serial aesthetics and practices in games and game cultures. Here is the abstract for the paper:

In this paper we are concerned to outline a set of perspectives, methods, and theories with which to approach the seriality of digital games and game cultures – i.e. the aesthetic forms and cultural practices of game-related serialization, which we see unfolding against (and, in fact, as a privileged mediator of) the broader background of medial and socio-cultural transformations taking place in the wake of popular media culture’s digitalization. Seriality, we contend, is a central and multifaceted but largely neglected dimension of popular computer and video games. Seriality is a factor not only in explicitly marked game series (with their sequels, prequels, remakes, and other types of continuation), but also within games themselves (e.g. in their formal-structural constitution as an iterative series of “levels” or “worlds”) as well as on the level of transmedial relations between games and other media (e.g. expansive serializations of narrative worlds across the media of comics, film, television, and games, etc.). Particularly with respect to processes of temporal “collapse” or “synchronization” that, in the current age of digitization and media convergence, are challenging the temporal dimensions and developmental logics of pre-digital seriality (e.g. because once successively appearing series installments are increasingly available now for immediate, repeated, and non-linear consumption), computer games are eminently suited for an exemplary investigation of a specifically digital type of seriality.

In the following, we look at serialization processes in digital games and game series and seek to understand how they relate to digital-era transformations of temporally-serially structured experiences and identifications on the part of historically situated actors. These transformations range from the microtemporal scale of individual players’ encounters with algorithmic computation processes (the speed of which escapes direct human perception and is measurable only by technological means) all the way up to the macrotemporal (more properly “historical”) level of collective brokerings of political, cultural, and social identities in the digital age. To account for this multi-layered complexity, we argue for a decidedly interdisciplinary approach, combining media-aesthetic and media-philosophical perspectives with the resources of discourse analysis and cultural history. We approach the seriality of digital games both in terms of textual and aesthetic forms as well as in the broader context of serialized game cultures and popular culture at large.

Please take a look and spread the word about the new issue of Eludamos. We would be more than happy to hear your feedback about our article, so feel free to leave a comment here. Enjoy!

(“Pixel Gnomes” image created by Shane Denson, based on hand-painted Mario & Luigi-style garden gnomes made by Karin Denson.)

Mediatization and Serialization

mediatization and serialization_poster.001

The very first thing I posted on this blog (in May 2011) was the above flyer, announcing a talk I was giving on the connections between “mediatization” and “serialization.” While sorting through some papers, I came across the text of the talk again and realized I still haven’t gotten around to doing anything with it. In the end, I have to admit that the concept of mediatization, as defined by the media sociologists and communications theorists I discuss in the talk, doesn’t really appeal to me that much. For one thing, I am much less concerned than they are to guard against charges of technological determinism; mediatization theory often seems preoccupied with keeping the human, or the social, in control (on some general problems with this preoccupation, see McKenzie Wark’s recent essay “Against Social Determinism”). However, a large part of the theoretical appeal of the notion of mediatization — as a process of change closely linked with processes like modernization and globalization — lies in its description of an apparent loss of autonomy, a ceding of human agency to the technical. And I think there’s something to be said for this: the past two-hundred some-odd years have witnessed an explosion of technical actants, placing us in ever more complex and opaque feedback loops with them and the environment they mediate to us: welcome to the anthropocene…

So this is not my problem with mediatization theory. Rather, my problem is with the insinuation that human agency was free of the taint of media or technics some two or three hundred (or however many) years ago, and that it only gradually became “mediatized.” My own notion of postnaturalism, summed up in the Latourian paraphrase that “we have never been natural,” is based on the idea of an essential and indissoluble (though by no means static) “anthropotechnical interface” that connects human and technical agencies in a transductive relation — there simply is no human agency without technical agencies, and vice versa. Still, it is necessary to take note of the empirical changes that take place against this cosmological horizon, and perhaps the notion of mediatization can be of service in this regard after all. If we set aside the worries over determinism, that is, and train our focus at a medium level of abstraction — somewhere between the cosmological and the phenomenologically/technically concrete and individual: at the level of supra-personal but not quite geological temporality or history — perhaps then an engagement with the concept of “mediatization” can help us think through the qualitative changes in agency that have taken place with the advent of the steam press, photography, film, radio, television, and digital media.

My talk on “Mediatization & Serialization” certainly does no more than scratch the surface in this regard, but in the hopes that it at least manages to do that, I have decided to reproduce the text here. As always, I am grateful for any comments!

Mediatization & Serialization

Shane Denson

(Note: This is a rough script of the talk I held at Leibniz Universität Hannover on May 18, 2011. Bibliographical info is missing, and footnotes are just placeholders.)

[Intro]

What I hope to do is to bring the concept of mediatization, which I’ll explain shortly, into dialogue with that of serialization, especially as it pertains to the seriality of modern popular entertainment (as exhibited in film and television series, for example). Most generally, the basis for this dialogue derives from the fact that both mediatization and serialization are, or so I contend, characteristically modern processes, perhaps even central to modernity itself. Mediatization, according to the term’s usage in recent communications theory and media sociology, is related to the fact that the modern world is witness to a consistent increase in the sheer number of technical media. These, in turn, are seen to be increasingly central to the shape and structure of sociocultural reality, so that it becomes increasingly untenable to think of media as a separate institution, exerting pressure on the others from outside. Instead, the situation shifts to one in which we find genuinely “mediatized” institutions, institutions transformed by media as an increasingly integral framing structure. So it no longer makes sense to speak of “the media and education” or “the media and the family,” as in traditional mass communications studies, but instead of thoroughly mediatized education, family, and so forth. Mediatization, according to Friedrich Krotz, is a “meta-process,” not itself empirically observable but, like the meta-processes of individualization, commercialization, and globalization, marking a real trajectory in the modern world and a force that qualitatively conditions the realm of empirically observable phenomena. In some respects, this view of mediatization as a meta-process might be compared to Friedrich Kittler’s idea of “media a priori,” but (against this association) it is constantly emphasized in these discussions that mediatization is to be seen as a non-deterministic process. As for serialization, this refers, like I said, to the popular seriality that explodes onto the scene in the nineteenth century with the advent of new media technologies (like the steam press, which enabled the rapid production of printed periodicals, including story papers, penny dreadfuls, and dime novels). Since then, serialized narrative forms have continued to proliferate alongside and in the media that would seem to have changed our worlds into properly mediatized worlds: in film, radio, television, and now digital media. The serialization of entertainment cannot, it would seem, be thought in isolation from the processes described by the concept of mediatization, and serialization may be thought of as a special case of the mediatization of leisure time, specifically as relates to media-inflected transformations of popular narrative. On the other hand, though, serialization is a truly special case of mediatization, in that it is a highly self-reflexive process: serial narratives observe changes in mediation, track them over time, and thus offer images of mediatization processes as they unfold—in “real time,” so to speak. Careful attention to serialization promises therefore to shed light on mediatization—and vice versa. What I call “techno-phenomenology,” and which I will come back to shortly, will help uncover the bidirectional communication between the two processes.

[Framing mediatization through serialization]

But first, to start making these connections concrete, I turn to some ideas and observations put forward by Frank Kelleter in a recent issue of Psychologie Heute. Kelleter notes that the evolution of TV sitcoms from the 1950s to today displays a continuous movement from the intact family as the site and occasion for humor, by way of the dysfunctional units of All in the Family or Married…with Children, to more recent sitcoms that revolve around a group of unrelated friends (as in Seinfeld, Friends, or Sex and the City). As Kelleter remarks, this development can be seen as a result of, and a reflection of, the increasing individualization of modern Western societies. Individualization, you recall, is one of the central meta-processes that, according to Friedrich Krotz, define modernity, along with the interrelated meta-processes of commercialization, globalization, and mediatization. And surely we can see these other processes reflected just as clearly in the globally syndicated, made-for-profit series that dominate TV screens today. But, as Kelleter recognizes, the reflection of social reality is only the beginning, and it would be shortsighted to reduce the link between serialization and mediatization to a passive commentary function that series can assume with respect to social developments.

On the contrary, serialized television actively frames our experience of the world: the week is punctuated at regular intervals by our favorite series, and our relations at home and on the job are perceived—though not uncritically—through the lenses of television drama, sitcoms, soaps, news, and police procedurals. Just as we judge series as either realistic or unrealistic, we inevitably compare reality and the people we encounter in the “real world” with the models we have met and occasionally become intimate with in the serialized world of television. In this respect, we can speak of a relatively direct (though hardly simple[1]) link between serialization and mediatization, as the serialized entertainment of TV transformatively reframes our interpersonal relations and behavioral expectations. Moreover, this activity of framing and reframing is itself increasingly the object of serial entertainment. The serial production and distribution of still incomplete narrative constructs opens a space for what Henry Jenkins calls “participatory culture,” and new social relations are created in the feedback loops between producers and consumers—social relations that are not only transformed but in fact generated through mediatization. And the media processes upon which these novel relations depend are increasingly the topic not only of fans’ discourses with one another but also of the serial productions themselves, which in the era of so-called Quality TV are aware of the existence of a well-informed, highly networked, and hardly passive fan base. Fans know how series are structured, and series’ producers know that fans know how they’re structured, and so the series themselves become increasingly complex and self-reflexive in response. The result is that the serialized communication that takes place in the inherently mediatized networks of producers and consumers of serial forms continues to proliferate in the manner of a self-serialization that takes as its object the serial maintenance of mediatized relations and communications in and through series. (And that only approximates the complex self-reflexive processes set in motion by a series like Lost.)

But while I’ve concentrated on recent television here, the link between serialization and mediatization is much broader. A moment ago, I invoked the figure of the frame to describe the work of regularly consumed TV series in organizing and transforming our experience, and I want to extend this to serial forms in other media as well. The figure of the frame has cropped up before in discussions of mediatization, often in connection with the idea of an overarching “media logic” (as the “orientation frame” or “processual framework” that increasingly structures social action).[2] But whereas this notion of media logic raises worries of determinism and is seen by some to cast mediatization as an implausibly linear process,[3] the sort of framing I have in mind is inherently reversible, volatile, and polyvocal—expressive of the non-neutral Prägkraft or “moulding forces” that communications scholar Andreas Hepp sees at work in media, while also compatible with his call (following David Morley) for a non-media-centric media theory capable of countenancing a central paradox of mediatization: capable, namely, of accounting for the increasingly central role of media in shaping our experience without thereby attributing the central or determinative causal agency of this development to media as a unified force. Divorced from the idea of a singular, teleological media logic, the figure of the frame illustrates what I call a process of “de/centering,” which Derrida exposes in a text ostensibly about painting. As he shows, frames are never simple or singular but highly unstable and phenomenally reversible entities. On the one hand, they stand outside the work, providing a background against which the framed content can emerge as a figure. On the other hand, though, the frame becomes part of the figure when seen against the background of the wall. Oscillating between ground and figure, the frame as margin or passepartout both centers and decenters the work: it enacts the multistable logic of de/centering that I propose explains the non-passive but non-deterministic framing function of media as revealed by a techno-phenomenological take on mediatization and serialization.

[Techno-phenomenology]

Techno-phenomenology is my term for an approach developed by American philosopher Don Ihde, following leads from the early Heidegger and Merleau-Ponty, among others. Generally speaking, techno-phenomenology looks at technologies neither as passive artifacts nor as determinative global systems, but instead as constituent parts of the relations that human agents maintain with their environments in concretely embodied, practical situations. What Ihde calls “mediating technologies” may occupy a variety of positions within the intentional relations of subjects to objects. For example, when one looks through a telescope, the device itself ideally disappears; it is, so to speak, “absorbed” into the perceiving subject’s sensorial apparatus to reveal far-away objects that couldn’t be seen with the naked eye. Prosthetically extending the embodied subject, the telescope occupies what Ihde calls an “embodiment relation.” In contrast to this, a radio telescope cannot be looked through in this direct manner; instead, its output must be looked at and interpreted, “read” as a sign of an objective reality that cannot be perceived directly. This type of technology instantiates a “hermeneutic relation”—occupying a semi-objective position as something to be looked at rather than through. But it is important to note that technologies, on this view, do not inevitably and irrevocably instantiate one or the other type of relation. As Heidegger’s famous hammer illustrates, an embodiment relation can always break down. And Don Ihde goes further to illustrate that breakage is not the only source of such reversal. Decontextualization, simple inexperience, or intentional aesthetic estrangement, for example, can all cause transformation, because techno-phenomenal relations are embedded in contexts of practice, which are in turn conditioned by social and cultural forces. The fact remains, though, that a “telic inclination,” as Ihde puts it, may inhere in a technology, predisposing a given technology to certain means of use and relation and not to others. Material factors exert pressures on praxis that push optical and radio telescopes, for example, to opposite ends of the spectrum of subject-object relations, while still allowing for non-typical forms of use or appropriation.

What goes for Ihde’s “mediating technologies” applies to media in a narrower sense as well. On a very general level, for example, we might say that textual media tend, for obvious reasons, towards hermeneutic relations, while the television screen tends to be looked through, rather than at, as a quasi-transparent window on the world. These are telic inclinations of the media—differential media logics in the plural, if you like—quite comparable to the non-absolute “tendencies” or “pressures” that Hepp terms the “moulding forces” of media. These forces, looked at from a techno-phenomenological perspective, can be seen to frame the perceptual and actional agencies of media users, but never in a univocal, absolute, or determinative manner. Significantly, contextual reversals remain a live possibility, as is demonstrated by the paradigmatic difference between early and so-called classical film. In the early years of the cinema, which Tom Gunning has defined as a “cinema of attractions” (1895-1905), the cinema itself was the main attraction; people went to see projection apparatuses, not films. But the display of cinematic magic and trick effects for their own sake came to be subordinated to narration, and the classical Hollywood style, which took shape by around 1917, worked to ensure the invisibility of narrative construction, rendering the apparatus of film a transparent window onto fictional worlds. Importantly, though, narrative serialization—in the form of film serials such as The Perils of Pauline, The Hazards of Helen, or The Exploits of Elaine—arose in the 1910s as a means of navigating the uncertain transitional phase between the early and classical paradigms. Staging neither the pure media spectacles of early attractions-style cinema nor the self-enclosed diegetic universes of classical film, these serialized story films vacillated between medial transparency and opacity, between narrative closure and an openness onto the non-diegetic conditions of their storytelling—both due to the incompleteness of the weekly episodes, which were segmented by cliffhangers and interrupted by the rhythms of the work week, and due to self-reflexive tendencies by which the serials ostentatiously demonstrated the medial means of an emerging form of narrative construction.[4]

[The nexus]

This brings me to the nexus that, as I see it, binds popular seriality and mediality. By mediality, I mean the fact and specific character or quality of a given process of mediation. Roger Hagedorn points to a special relation between seriality and mediality when he observes that serial narratives often “serve to promote the medium in which they appear” (5). Medial self-reflexivity is, then, in a sense a natural facet of the serial form’s role in helping “to develop the commercial exploitation of a specific medium” (5). The serialized novels of the nineteenth century feuilleton helped sell newspapers, and color comic strips advertised newly developed four-color printing processes. Early radio and television series served to attract consumers to the new media, to induce them to purchase expensive devices, and then to hook them with ongoing stories and recurring entertainments. Popular seriality is thus closely linked with the development of modern media, or with a media modernity characterized by constant pressure towards media-technological innovation. Serialization is correlated, that is, with the quantitative and qualitative changes which lay the very groundwork for the meta-process of mediatization. Concerned with their own medial forms, as well as the transformation of the media landscape implied by the emergence of new, competing media, series probe, expose, demonstrate, and experiment with their own mediality and compare various media with one another; moreover, because series unfold over time, they are capable of tracking the medial processes and changes upon which the meta-process of mediatization depends or supervenes.

Especially at transitional moments of media change, a techno-phenomenological perspective on seriality reveals a privileged view of the processes that are basic to mediatization; conversely, the concept of mediatization illuminates the lower-level work of serialization in the broader perspective of modernity. For example, late nineteenth century dime novels can be approached as medially inconspicuous channels through which simple, formulaic stories of frontier heroes, urban detectives, and young inventors were told; but when we see these tales in relation to the innovations in print technologies that made their production possible, to the transcontinental rail systems upon which their distribution relied, and to the increasingly urbanized settings in which their readers lived and worked, then the serially implemented locomotives, telegraphs, and other communication and transportation technologies that fill the pages of these stories seem to belong less to their narrative worlds than to the extra-diegetic machinery of their mediation and consumption. In techno-phenomenological terms, we find here a radical ambivalence between the transparency of embodiment relations and the medial opacity of hermeneutic relations. The serial form, which oscillates formally between repetition and variation, stages lifeworld changes in this unstable or “de/centered” manner and discloses thereby the underlying mechanics of mediatization in the process of its occurrence.

And because the specific periodicity of serialized forms can vary widely, ranging from daily to weekly to longer-term intervals, the perspective that series offer on mediatization varies accordingly. In addition to linear or episodic series that unfold within a single medium over a short period of time, there are also plurimedial series that may be staged over the course of decades. A serially staged figure such as Frankenstein’s monster constitutes such a series: originating in a novel, appearing numerous times on theater stages, achieving iconic form on film, and continuing to proliferate in comics, on TV, and in video games, the monster tracks virtually the entire course of modern media history. And it thrives particularly at moments of media change, such as the sound-film transition that ironically gave birth to Karloff’s mute monster. Robbed of the articulate speech acquired by the monster of the novel, the monster of the movies briefly served to highlight the fact and the eerie quality of sound film’s novel mediality, exemplifying the reversible logic of the frame by foregrounding the medial infrastructure over the mediated narrative. This reversal of figure and ground operated on the basis of the monster’s serial staging, on the basis of a preexisting familiarity and recognizability rendered strange in the new medium. With the habituation of sound film, the strangeness wore off and the iconic monster came to serve as the baseline for an ongoing serialization process. Here, if we look carefully, we can see the procession of major and minor media transformations that have made our world a properly mediatized one.

Finally, we might recognize here a meta-serial development, a historical shift in the constitution of serial forms, which provides a sort of wide-angle view of mediatization, as if through the wrong end of a telescope. In the procession from linear storytelling in a single medium, to the serialized proliferation of narratives repeated and varied in a fragmented plurimedial frame (such as is embodied by Frankenstein’s monster), to the recent advent of transmedial storytelling in the wake of a digitally induced media convergence, we see a long-term transformation of serial forms that speaks to an experience of progressive deterritorialization. Serially recurrent characters became dislodged from their material ecospheres, i.e. from the media in which they were born, through the proliferation of competing medial forms that vie for our attention. Likewise, we have been uprooted or liberated (depending on your perspective) from our immediate surroundings and, through these very media, been put in touch with distant, spatially nonlocal, communities. But a de/centered view of this development is neither linear nor certain in its assessment of the outcome. Transmedial storytelling continues the trend of displacement with respect to a singular or stable medial framework, but it reverses the diegetic fragmentation exemplified by Frankenstein’s monster and other serial icons of the twentieth century. Today, in series that span the media of television, film, print, and digital media, we find new tendencies toward reterritorialization, staged, though, as a complex and hardly straightforward affair. There is a renewed interest in creating unified diegetic worlds, in spite of or precisely because of the multiplicity of medial frames somewhat euphemistically united in talk of convergence. It is unclear what this says about us and our contemporary serialized experience of mediatization, but I think we can rule out the idea that today’s transmedial series simply “reflect” social reality. Instead, they frame our experience in a de/centered manner that both displays and enacts a central paradox of our mediatized worlds: media are increasingly central in structuring our experience, but there is a reversible margin from which these structures remain open to negotiation. As I have tried to show, this reversibility is essential to the process of serialization, thus ensuring the continued importance of popular seriality as a site of the non-deterministic production of a mediatized modernity.


[1] Indeed, complexification is central to Kelleter’s argument, which is also sensitive to the self-reflexive functions of serialization which I exploit in this paper. See also Jason Mittell on “narrative complexity.”

[2] The notion of the frame, popular of late in a variety of discourses and disciplines, also crops up regularly in discussions of mediatization, and I think a great deal hinges on our understanding and use of the concept. For example, in an essay that approaches mediatization in terms of the “institutionalization of media logic” (42), Andrea Schrott invokes the figure of the frame to explain this media logic, which she describes as an “orientation frame” (48) that is non-neutral with respect to content. This accords with Altheide and Snow’s own description of media logic, in their 1979 book of that title, as “a processual framework through which social action occurs” (15). Similarly, Norm Friesen and Theo Hug invoke the notion of “framing” to signal the “epistemological orientation” increasingly provided by media (80), or what they call “the role of the mediatic a priori” (79).

[3] Nick Couldry’s worry.

[4] Moreover, formal changes are coupled with social changes (gender, class), and these are highly relevant to a study of the mediatization of entertainment or of leisure time, as well as (and correlated with) a mediatization of identity or social identification. With respect to the specific example of transitional-era film serials (and serial-queen melodramas), see my “The Logic of the Line Segment: Continuity and Discontinuity in the Serial-Queen Melodrama” (forthcoming in Serialization in Popular Culture, edited by Robert Allen and Thijs van den Berg, New York: Routledge, 2014).