Post-Cinematic N-Grams

2016-05-01 07.23.33 pm

Advertisements

Speculative Data: Full Text, MLA 2016 #WeirdDH

SpeculativeData-jpg.001

Below you’ll find the full text of my talk from the Weird DH panel organized by Mark Sample at the 2016 MLA conference in Austin Texas. Other speakers on the panel included Jeremy Justus, Micki Kaufman, and Kim Knight.

***

Speculative Data: Post-Empirical Approaches to the “Datafication” of Affect and Activity

Shane Denson, Duke University

A common critique of the digital humanities questions the relevance (or propriety) of quantitative, data-based methods for the study of literature and culture; in its most extreme form, this type of criticism insinuates a complicity between DH and the neoliberal techno-culture that turns all human activity, if not all of life itself, into “big data” to be mined for profit. Now, it may sound from this description that I am simply setting up a strawman to knock down, so I should admit up front that I am not wholly unsympathetic to the critique of datafication. But I do want to complicate things a bit. Specifically, I want to draw on recent reconceptions of DH as “deformed humanities” – as an aesthetically and politically invested field of “deformance”-based practice – and describe some ways in which a decidedly “weird” DH can avail itself of data collection in order to interrogate and critique “datafication” itself.

SpeculativeData-jpg.002

My focus is on work conducted in and around Duke University’s S-1: Speculative Sensation Lab, where literary scholars, media theorists, artists, and “makers” of all sorts collaborate on projects that blur the boundaries between art and digital scholarship. The S-1 Lab, co-directed by Mark Hansen and Mark Olson, experiments with biometric and environmental sensing technologies to expand our access to sensory experience beyond the five senses. Much of our work involves making “things to think with,” i.e. experimental “set-ups” designed to generate theoretical and aesthetic insight and to focus our mediated sensory apparatus on the conditions of mediation itself. Harnessing digital technologies for the work of media theory, this experimentation can rightly be classed, alongside such practices as “critical making,” in the broad space of the digital humanities. But due to their emphatically self-reflexive nature, these experiments challenge borders between theory and practice, scholarship and art, and must therefore be qualified, following Mark Sample, as decidedly “weird DH.”

SpeculativeData-jpg.003.jpeg

One such project, Manifest Data, uses a piece of “benevolent spyware” that collects and parses data about personal Internet usage in such a way as to produce 3D-printable sculptural objects, thus giving form to data and reclaiming its personal value from corporate cooptation. In a way that is both symbolic and material, this project counters the invisibility and “naturalness” of mechanisms by which companies like Google and Facebook expropriate value from the data we produce. Through a series of translations between the digital and the physical—through a multi-stage process of collecting, sculpting, resculpting, and manifesting data in virtual, physical, and augmented spaces—the project highlights the materiality of the interface between human and nonhuman agencies in an increasingly datafied field of activity. (If you’re interested in this project, which involves “data portraits” based on users’ online activity and even some weird data-driven garden gnomes designed to dispel the bad spirits of digital capital, you can read more about it in the latest issue of Hyperrhiz.)

SpeculativeData-jpg.004

Another ongoing project, about which I will say more in a moment, uses data collected through (scientifically questionable) biofeedback devices to perform realtime collective transformations of audiovisual materials, opening theoretical notions of what Steven Shaviro calls “post-cinematic affect” to robustly material, media-archaeological, and aesthetic investigations.

SpeculativeData-jpg.005

These and other projects, I contend, point the way towards a truly “weird DH” that is reflexive enough to suspect its own data-driven methods but not paralyzed into inactivity.

Weird DH and/as Digital Critical (Media) Studies:

So I’m trying to position these projects as a form of weird digital critical (media) studies, designed to enact and reflect (in increasingly self-reflexive ways) on the use of digital tools and processes for the interrogation of the material, cultural, and medial parameters of life in digital environments.

SpeculativeData-jpg.006

Using digital techniques to reflect on the affordances and limitations of digital media and interfaces, these projects are close in spirit to new media art, but they are also apposite with practices and theories of “digital rhetoric,” as described by Doug Eyman, with Gregory Ulman’s “electracy,” or with Casey Boyle’s posthuman rhetoric of multistability, which celebrates the rhetorical affordances of digital glitches in exposing the affordances and limitations of computational media in the broader realm of an interagential relational field that includes both humans and nonhumans. In short, these projects enact what we might call, following Stanley Cavell, the “automatisms” of digital media – the generative affordances and limitations that are constantly produced, reproduced, and potentially transformed or “deformed” in creative engagements with media. Digital tools are used in such a way as to problematize their very instrumentality, hence moving towards a post-empirical or post-positivistic form of datafication as much as towards a post-instrumental digitality.

SpeculativeData-jpg.007

Algorithmic Nickelodeon / Datafied Attention:

My key example is a project tentatively called the “algorithmic nickelodeon.” Here we use consumer-grade EEG headsets to interrogate the media-technical construction and capture of human attention, and thus to complicate datafication by subjecting it to self-reflexive, speculative, and media-archaeological operations. The devices in question cost about $100 and are marketed as tools for improving concentration, attention, and memory. The headset measures a variety of brainwave activity and, by means of a proprietary algorithm, computes values for “attention” and “meditation” that can be tracked and, with the help of software applications, trained and supposedly optimized. In the S-1 Lab, we have sought to tap into these processes in order not just to criticize the scientifically dubious nature of these claims but rather to probe and better understand the nature of the automatisms and interfaces taking place here and in media of attention more generally. Specifically, we have designed a film- and media-theoretical application of the apparatus, which allows us to think early and contemporary moving images together, to conceive pre- and post-cinema in terms of their common deviations from the attention economy of classical cinema, and to reflect more broadly on the technological-material reorganizations of attention involved in media change. This is an emphatically experimental (that is, speculative, post-positivistic) application, and it involves a sort of post-cinematic reenactment of early film’s viewing situations in the context of traveling shows, vaudeville theaters, and nickelodeons. With the help of a Python script written by lab member Luke Caldwell, a group of viewers wearing the Neurosky EEG devices influence the playback of video clips in real time, for example changing the speed of a video or the size of the projected image in response to changes in attention as registered through brain-wave activity.

At the center of the experimentation is the fact of “time-axis manipulation,” which Friedrich Kittler highlights as one of the truly novel affordances of technical media, like the phonograph and cinema, that arose around 1900 and marked, for him, a radical departure from the symbolic realms of pre-technical arts and literature. Now it became possible to inscribe “reality itself,” or to record a spectrum of frequencies (like sound and light) directly, unfiltered through alphabetic writing; and it became possible as well to manipulate the speed or even playback direction of this reality.

SpeculativeData-jpg.009

Recall that the cinema’s standard of 24 fps only solidified and became obligatory with the introduction of sound, as a solution to a concrete problem introduced by the addition of a sonic register to filmic images. Before the late 1920s, and especially in the first two decades of film, there was a great deal of variability in projection speed, and this was “a feature, not a bug” of the early cinematic setup. Kittler writes: “standardization is always upper management’s escape from technological possibilities. In serious matters such as test procedures or mass entertainment, TAM [time-axis manipulation] remains triumphant. [….] frequency modulation is indeed the technological correlative of attention” (Gramophone Film Typewriter 34-35). Kittler’s pomp aside, his statement highlights a significant fact about the early film experience: Early projectionists, who were simultaneously film editors and entertainers in their own right, would modulate the speed of their hand-cranked apparatuses in response to their audience’s interest and attention. If the audience was bored by a plodding bit of exposition, the projectionist could speed it up to get to a more exciting part of the movie, for example. Crucially, though: the early projectionist could only respond to the outward signs of the audience’s interest, excitement, or attention – as embodied, for example, in a yawn, a boo, or a cheer.

SpeculativeData-jpg.010

But with the help of an EEG, we can read human attention – or some construction of “attention” – directly, even in cases where there is no outward or voluntary expression of it, and even without its conscious registration. By correlating the speed of projection to these inward and involuntary movements of the audience’s neurological apparatus, such that low attention levels cause the images to speed up or slow down, attention is rendered visible and, to a certain extent, opened to conscious and collective efforts to manipulate it and the frequency of images now indexed to it.

According to Hugo Münsterberg, who wrote one of the first book-length works of film theory in 1916, cinema’s images anyway embody, externalize, and make visible the faculties of human psychology; “attention,” for example, is said to be embodied by the close-up. With our EEG setup, we can literalize Münsterberg’s claim by correlating higher attention levels with a greater zoom factor applied to the projected image. If the audience pays attention, the image grows; if attention flags, the image shrinks. But this literalization raises more questions than it answers, it would seem. On the one hand, it participates in a process of “datafication,” turning brain wave patterns into a stream of data called “attention,” but whose relation to attention in ordinary senses is altogether unclear. But this datafication simultaneously opens up a space of affective or aesthetic experience in which the problematic nature of the experimental “set-up” announces itself to us in a self-reflexive doubling: we realize suddenly that “it’s a setup”; “we’ve been framed” – first by the cinema’s construction of attentive spectators and now by this digital apparatus that treats attention as an algorithmically computed value.

So in a way, the apparatus is a pedagogical/didactic tool: it not only allows us to reenact (in a highly transformed manner) the experience of early cinema, but it also helps us to think about the construction of “attention” itself in technical apparatuses both then and now. In addition to this function, it also generates a lot of data that can indeed be subjected to statistical analysis, correlation, and visualization, and that might be marshaled in arguments about the comparative medial impacts or effects of various media regimes. Our point, however, remains more critical, and highly dubious of any positivistic understanding of this data. The technocrats of the advertising industry, the true inheritors of Münsterberg the industrial psychologist, are anyway much more effective at instrumentalizing attention and reducing it to a psychotechnical variable. With a sufficiently “weird” DH approach, we hope to stimulate a more speculative, non-positivistic, and hence post-empirical relation to such datafication. Remitting contemporary attention procedures to the early establishment of what Kittler refers to as the “link between physiology and technology” (73) upon which modern entertainment media are built, this weird DH aims not only to explore the current transformations of affect, attention, and agency – that is, to study their reconfigurations – but also potentially to empower media users to influence such configuration, if only on a small scale, rather than leave it completely up to the technocrats.

The Gnomes Are Back: Business cARd 2.0

gnome-cARd

Ever since our old AR platform was bought out and shut down by Apple, the “data gnomes” that Karin and I developed in conjunction with the Duke S-1: Speculative Sensation Lab’s “Manifest Data” project have been bumbling about in digital limbo, banished to 404 hell. So today I finally made the first steps in migrating our beloved creatures over to a new AR platform (Wikitude), where they’re starting to feel at home. While I was at it, I went ahead and reprogrammed my business card:

2016-01-31 12.21.55 pm

The QR code on the front now redirects the browser to shanedenson.com, while the AR content on the back side is made visible with the Wikitude app (free on iOS or Android) — just search for “Shane Denson” and point your phone/tablet’s camera at the image below:

2016-01-31 12.22.20 pm

(In case you’re wondering what this is: it’s a “data portrait” generated from my Internet browsing behavior. You can make your own with the code included in the S-1 Lab’s Manifest Data kit.)

Opening of Hyperrhiz: Kits, Plans, and Schematics. An Exhibition of Electronic Art

The New Krass

Gnomes at Hyperrhiz1Hyperrhiz exhibit

The first two pictures from the opening of the Hyperrhiz exhibit at the Digital Studies Center at Rutgers University-Camden. Opening Reception October 15, 4-6pm.

Six Data gnomes, data portraits, and other physical and augmented elements of Manifest Data, a project of the Duke S-1 Speculative Sensation Lab (with Amanda Starling Gould, Karin & Shane Denson, Luke Caldwell, Libi Striegl, and David Rambo), will be on display alongside other contributions to a special issue of Hyperrhiz on “Kits, Plans, and Schematics.”

View original post

Making Mining Networking: Video Documentation

Above, some video documentation of the pieces included in Making Mining Networking, the exhibition that Karin and I have going on until September at Duke University. As I posted recently, the augmented reality platform we used to make the interactive components (Metaio) has been sold to Apple and will be going offline at the end of the year. All the more reason to document everything now — but until December 15 you can still try out the pieces yourself, either in person at the exhibition or on your own computer screen with a smart device (see the images here)!

The (generative, network-driven) music is from the project “Listen to Wikipedia,” by Hatnote — which seemed a perfect match for the theme of Making Mining Networking!

Making Mining Networking: Exhibition Extended, But Not For Long…

Tutorial_Level

Making Mining Networking went on display on April 20, 2015 at Duke University’s The Edge digital workspace. The show was originally scheduled to run until mid May, but it was extended several times until September 2015. After that, a few of the pieces are slated to be shown in Fall 2015 in an exhibit organized by the online journal Hyperrhiz: New Media Cultures and the Digital Studies Center at Rutgers University Camden.

In the meantime, however, it was reported in late May that the augmented reality platform we used to build the interactive components of our pieces had been sold – effectively putting an expiration date on our artworks. Metaio GmbH, makers of the popular Junaio AR browser and the underlying engine that allowed us to augment our QR-based paintings with videos, 3D objects, and HTML hyperlinks, were acquired by one of the biggest corporations in the business: Apple.

Even before the buyout was confirmed, rumors had started circulating after Metaio abruptly closed their community forums, cancelled their annual developers’ convention, and stopped selling their software and services. An ominous message went up on the Metaio website (metaio.com):

Metaio products and subscriptions are no longer available for purchase.

Downloads of your previous purchases will be available until December 15th, 2015, and active cloud subscriptions will be continued until expiration. Email support will continue until June 30th, 2015.

Thank you.

(No, thank you!) Lacking any explanation, users of the Metaio/Junaio AR platform were left to speculate about the future of their advertising campaigns, educational applications, and (as in our case) artworks.

Shortly thereafter, after the acquisition by Apple came to light, our worst fears were confirmed in the FAQs section on junaio.com. There we read:

Channel publishing to Junaio is no longer available. All existing channels will continue to be available until December 15th, 2015.

In other words, the pieces included in Making Mining Networking will no longer be functional at the end of the year. The QR codes painted on these canvases will no longer work; the pieces will then be flattened from the interactive physical/virtual assemblages they were designed to be and rendered into … paintings. Or worse, they will retain an executable dimension, albeit a non-operational one, and it will be supplemented by a weirdly representational dimension: effectively, these will then be paintings of 404 error codes.

MarxMarkovPainting-frame

Like all of the pieces, “The Magical Marx-Markov Manifesto Maker” is therefore destined to lose its magic; the QR code, when scanned with a smartphone, will lead the user’s browser into virtual nothingness. On the other hand, though, pieces like “The Gold Standard” and “Gnomecrafting” might still have something to tell us – precisely because their non-operationality will render visible the inevitable entanglement of proprietary platforms and obsolescent objects that is the material heart and soul of digital capital.

goldstandard-coins_frame

Making Mining Networking is (or was?) about probing the borders between the virtual and the physical – boundaries that are inscribed in stone (e.g. rare earths) as much as they are written in code. With our works, we have sought to invite users to experiment with this interface, opting for a playful approach to a space that we know is about deadly serious transactions (in the realms of capital and of the environment, to begin with). We installed our data gnomes at the physical/virtual border where they stood as talismans to ward off the bad spirits of digital capital – but we were never so naïve as to believe that they could really protect us for long. We still believe that we regained something of personal value by reclaiming our data from corporate mining and making something weird and inscrutable with it, but now a corporate transaction is about to render our productions invisible.

gnomecrafting_small

Again, however, it is the seeming totality of such corporate power to make things invisible – to make all that’s solid melt into zeroes and ones – that is paradoxically made visible at this juncture, where links are inoperative, QR codes are non-functional, and paintings are not just paintings but paintings of such failure.

The_9-small

Data-Portrait2 Data_Portrait1

Following our initial disappointment, then, we now now eagerly await the appointed date, our “doomsday” of December 15, 2015 – when the truth of Making Mining Networking will be revealed. Will it be a simple 404 message, or can we hope for something else to make manifest the physical/virtual interface as it exists in our era of climate change, high-speed finance, and the biopolitical mining of all that breathes and lives? Only time will tell…

In the meantime, these works exist as reminders of the expiration date that is implicitly inscribed in all of our devices – and, potentially, our very planet – at the hands of global digital capital. Play with them, think with them, experience with them – and await with us their obsolescence…

IMG_9422a

The Gnomes of #Netcologies

The New Krass

Gomes-at_the_Edge_small

Karin + Shane Denson
20″ x 20″
Acrylic on canvas

This painting unlocks seven geolocated augmented reality (AR) gnomes that you can discover outside the building. Just follow the brief instructions that appear on your device after scanning the QR code. Enjoy your walk and share your screenshots on twitter with the hashtag #netcologies.

IMG_4678IMG_4687IMG_4689IMG_4729

IMG_8951Unbenannt-1IMG_8945IMG_4722

View original post