Seeking Improvisation on the Open Web Platform
Haig Armen – Emily Carr University of Art & Design
John Maxwell – Simon Fraser University
Kate Pullinger – Bath Spa University
Abstract
Teaching creativity in digital media comes with its challenges. Obvious questions arise: what is the most effective creative process? what tools should be used? what is the right format? As Maslow’s Hammer reminds us, if you only have a hammer, everything looks like a nail; perhaps the challenge is that of perception. In this article we reflect on an experimental digital fiction workshop that took place in the summer of 2014. The workshop aimed to explore the boundaries of digital fiction using open web technology. The article discusses how process and tools forge the pathways and ultimately shape our outcomes. As we work to establish fully digital workflows and widely-adopted tools for creating digital narratives, we inadvertently limit our possibilities and fall into producing the same formats and modes of interaction over and over. In this nascent form, digital fiction requires efforts aimed for the fringes, to explore the boundaries of medium and genre. Today’s software authoring tools are designed for carefully composing and organizing content, not collaboratively improvising to create new forms of content, navigation and experience. Without a better understanding of the materiality of digital media gained through code proficiency and improvising on the form, we are limiting our ability to achieve more sophisticated forms of expression.
The universe of improvisation is constantly being created; or rather, in each moment a new universe is created… At any moment, an event may occur for no reason at all, with no relation at all to the preceding event… In this universe each moment is an entelechy, with both its cause and its end contained in itself.
Introduction
In June, John Maxwell, Kate Pullinger and I ran a week-long digital fiction workshop hosted by Simon Fraser University in Vancouver, BC. The experimental workshop was billed as a way to explore collaborative writing and production of digital fiction, inspired and directed by Kate Pullinger, a pioneer of the form.
Inspired by a conversation we had at Books in Browsers 2013 and a mutual admiration of Adam Hyde’s BookSprints , the idea of collaborating on a digital book workshop was born. Apart from gathering people together for an intensive limited timeframe to produce an outcome, this workshop had little resemblance to an actual BookSprint. We liked the idea of creating a framework – planning the activities and the technologies we’d use around creating a piece of digital fiction in a tight, high-intensity collaborative environment.
Setting the Stage
As the Digital Fiction workshop became a reality in the spring of 2014, we prepared by establishing the structure, content and technology for the event. Months before the scheduled workshop John and Haig discussed establishing a structure for the week of the gathering. Yet not knowing exactly what type of people the workshop would attract made it difficult to forge a strict structure. We wanted to bring together a group of interested authors and creators to participate in a real-time collaborative effort, writing and designing a multi-modal work over a short period of time. So we needed to carefully scaffold things to allow our workshop participants to move forward quickly.
John and I saw eye to eye about the kind of tools to provide to our workshop participants. For perhaps the first time in the history of publishing there is a common platform and software ecosystem that is free and open to all. The wildly popular technologies underpinning the Open Web Platform act as an alternative to the specialized, expensive toolsets that have dominated publishing (Linotype, InDesign, even Flash), the open web lets the finely tuned skills and sensibilities of authors, designers, and producers be decoupled from the exclusivity of proprietary tools. We both see Open Web technology as the only probable and preferable way forward.
In the weeks leading up to the workshop, we created a workflow that began with a wiki as a central collaborative writing hub where participants of the workshop were able to write in plain text, markdown, and HTML markup: markdown for formatting text and HTML to specify code for differentiating data types and insert media elements.
The next step was to find a way to generate HTML from the wiki. After a number of attempts John settled on using Pandoc, John MacFarlane’s free command-line tool for converting just about any flavour of markup and markdown. The final piece of our workflow was our target production framework, Caleb Troughton’s excellent Deck.js, a jQuery library designed for slide presentations that proved to be both flexible and extensible. The Deck.js library gave us the ability to create visual transitions from slide to slide, trigger html elements to animate across the screen and control audio/video—all of this, in an elegant, full-screen presentation that was compatible across multiple browsers and platforms.
Our Plan
Our intended workflow was to have our workshop participants begin sketching and storyboarding on sticky notes, index cards and whiteboards, then move onto laptops, from writing in markdown on our wiki to HTML5 via Pandoc. In the wiki, we could write a “script” for each story segment, including not only the text, but the images and audio as well. The wiki allowed everyone in the group to edit, make quick changes, create new segments, and quickly see and click through their results in a browser window.
John and I worked out extensions to Deck.js to facilitate audio and video triggers at different places, a global navigation system, and some custom effects for builds. The idea was to keep the gory details of the HTML5 production contained in an iterative production pipeline, in order to o facilitate a group of writers working out the story and how to tell it, letting them work through it in an agile, iterative way.
Observations
We gathered 12 people in June 2014 for the workshop: Kyle Carpenter, Alexandra Caufin, Jodie Childers, Jennifer Delner, Bob Fletcher, Rochelle Gold, Nicola Harwood, Inba Kehoe, Shazia Ramji, Kaitlyn Till, Jessica Tremblay. They were mostly graduate students and academics, but also writers, editors and photographers.
As Adam Hyde has pointed out, to run a successful Book Sprint you need: good people, a good venue, good food and an experienced facilitator. We had just two of these: a good venue and good people. We didn’t have the budget for catering and we were certainly not experienced at this sort of engagement!. What we did have was 15 hungry people in a room with lots of computers and technology on tap, and paper of all sizes.
Over the space of a week we came together on a storyline, a script, imagery, and ideas for animation and audio. Our workshop goal was to create, experiment and discover new tools, to explore new methods, to collaborate ‘digitally’, to forge new territory. Collectively we called what we created The Last Cartographer, a ‘neo-diluvian’ tale of love and loss, set in near-future Vancouver. By the end of the week-long session, we had a five-part, not-quite-linear narrative, mostly created using text and images, but with audio, video, and animation elements as well. (fig-0-LastCartographer.png – “The Last Cartographer’s opening screen”)
As a separate effort, The Last Cartographer was later further developed by my 3rd. Year design students and I at Emily Carr University of Art & Design in September. There are now 5 versions of the Cartographer story, based on the same story, but differing in visual and interactive treatments.
Working on paper
We liked to say that The Last Cartographer was produced entirely using free open web technologies. That was our intention but the reality is that this piece was produced mostly on paper. (fig-1-Collaborating using paper)
Our participants largely worked on paper, whether on stickies, flipcharts or index cards and whiteboards. They drew and talked, wrote and talked. In fact, probably the bulk of the week was spent sketching on paper. Why? Because paper affords extremely quick, easy, social development of ideas.
(fig-2-cardsorting.jpg – “Moving story fragments around on paper”)
It’s also worth mentioning that the workshop attracted people who identified as writers, not designers, nor coders. But despite that, they mostly produced sketches and storyboards, rather than any amount of text. They thought primarily in terms of a small bit of text in the foreground and a background image, much like a picture book. Where did this model come from? Was it from Kate’s influence via Flight Paths? Or perhaps presenting my work on CBC Radio 3 magazine during the week? Or something else?
(fig-4-sequencing.jpg “Storyboarding, sequencing and visualizing”)
We’ve all read our fair share of picture books. Could this pattern be so ingrained that it seeps into our thoughts of onscreen fiction inconspicuously enabling us to fall into the patterns – the modes of storytelling – that are most comfortable and familiar to us? John and I provided a straightforward production workflow but it wasn’t fluid enough to really allow our writers to work creatively within it. The ideas were being generated outside of the tools.
Jamming it out: The Wolf
In the weeks before the workshop, while John and I were working out how we would scaffold things together, and while we were evaluating frameworks, we threw together a little story about a lonely wolf, using Deck.js. The idea was mostly to test out what we were doing: image, text, placement, backgrounds, masking. John created a few screens, I created a few, John added more, etc. We bounced a story back and forth between us. It was a jam session, akin to two musicians tuning up our instruments and warming up. Between the two of us we have something approaching 40 years of web development experience; we improvised our Wolf story, in markdown, raw HTML, CSS, jQuery, while ‘tuning up’. In contrast, in our June workshop, we improvised using paper: on stickies and flipcharts, in words, in sketches and in arrangements.
Reflections
With hindsight we are now able to look critically on our Digital Fiction Workshop experience. We see three main points of reflection. First, the role of proficiency in creating digital fiction. Second, how profoundly the tools shape our outcomes. And third, the important distinction between composing and improvising in the context of creating digital media.
The Role of Proficiency
Our workshop participants mostly defaulted to creating on paper and pencil. Although they were all proficient with digital tools, when it came to creating collaboratively, paper simply didn’t get in the way of their ideas and creative process. Paper does a surprisingly good job of enabling fast, iterative idea development—something not lost on designers, nor software developers.
We also put software in front of our writers: easy to use tools like wikis. Yet software only gets out of the way of one’s creative inspiration when you know it intimately. This is something that John and I were able to do, jamming through our improvised Wolf story, because of many years of coding, designing, and web development. Chimero puts it well, “Let’s talk about making tools. The things we make should either reduce pain, increase pleasure, or do some mix of the two.” Yet this was not an experience open to our workshop participants. Fortunately, they found ways around it, photographing their storyboards and sequencing their narratives in a variety of crafty ways that often approached but perhaps never quite reached final resolution.
In our talk at Books in Browsers 2013, we spoke about the craft of publishing and the importance of understanding the grain or materiality of digital media. With a better understanding of materiality we are able to craft with nuance in mind. Taking advantage of the idiosyncrasies of the medium, to create more sophisticated forms of expression, forms that are appropriate for and unique to digital narratives.
In the workshop, there was a comfort with exploring and experimenting with characters, setting, themes and plot. This is where most of the improvisation occurred. Our group was less able to experiment with the presentation level with software. We believe that with a stronger understanding of the possibilities that come with digital media and some solid coding skills, our group would have been able to jam out some amazing concepts directly in the digital realm.
Tools & Frameworks
In our workshop we gave our participants tools that we thought would be open and malleable in shaping a digital piece of work. Yet even these loosely connected open web tools implied a very specific way to present content. To move beyond the prescribed way that, for instance, Deck.js wants to be used requires a level of comfort and proficiency with software of that kind. It appears that the tools we chose clearly dictated the type of narrative that would be produced. Many of the commercially available software tools for creating digital books are just as limiting.
My colleague Celeste Martin at Emily Carr University of Art + Design has catalogued the various interaction patterns that a majority of electronic books follow . Each pattern clearly details a style of navigation & content delivery – essential learning for designers with digital books. Yet these rudimentary patterns are directly linked to specific software platforms. Each platform is optimized for a specific interaction pattern and we are not easily able to push beyond the prescribed navigation and content presentation paradigms.
(fig-6-ebook-patterns.png – Celeste Martin’s Book Patterns)
Software can transcend mere utility when it allows its user to simultaneously think expansively while providing abstract constraints that help organize and build mental models. Consider how Jazz musicians use the harmonic structures of standard jazz repertoire as a shared foundation to improvise upon:
Many of the most popular jazz compositions — the standards — are repeatedly transcribed and compiled into Real Books and often used as learning tools. Real Books, as well as their many variations (Fake, Latin Jazz, Jazz Rock and, latterly, iReal Books), provide conventional harmonic sequences and phrase components that are acquired and employed as parts of each new musician’s improvisational complex vocabulary.
Just as Haftor Medbøe explains the importance of creating frameworks for Jazz musicians to build shared understanding and foundational structure for improvisational collaboration. Liz Danzico also described how designers of software could approach improvisation as a goal:
“Just as Miles Davis created a new form of jazz that allowed a new generation of musicians to play beyond themselves, so do we have the opportunity to create frameworks for audiences to create in realtime.”
Apart from iBooks Author, all the other platforms require a dual mode of creation: compose then preview, compose then preview, and repeat. This switching of modes is slow and tedious and makes the tools too opaque. Where as the visual (direct manipulation) tools are the ones that seem to melt away, becoming transparent and allowing for improvised moments. How can we seriously approach creating digital fiction without having (a) a mature, visual, Direct Manipulation toolkit, or (b) ten years or more experience with the code so that you can visualize what your code will produce before you see it rendered? The latter seems to be what experienced web developers do, thinking in code but imagining what appears in the browser. It’s a mode that clearly works, but it seems to limit creative engagement.
Composing & Improvising
Composing, in music, requires that same kind of abstract sense, where you can imagine the orchestra playing the notes you write. In the same sense, coding with web technology requires you to imagine how the browser will render your code. Most of the software we use on our personal computers today are designed for composition not improvisation. Yet improvisation is critical if digital fiction is ever going to be art.
As Alan Kay puts it,
There is the desire of a consumer society to have no learning curves. This tends to result in very dumbed-down products that are easy to get started on, but are generally worthless and/or debilitating. We can contrast this with technologies that do have learning curves, but pay off well and allow users to become experts (for example, musical instruments, writing, bicycles…)
We seem to be always striving to improvise with consumer level software but unable to reach interesting results because those tools support general patterns and conventions rather than the deep engagement that facilitates improvisation. Alan Kay urged us to improvise on the early generations of the personal computer. He wanted us to build our own tools that would allow us to put things together dynamically.
Designer Bruce Mau also suggested we make your own tools in his Incomplete Manifesto for Growth: “Hybridizing your tools in order to build unique things. Even simple tools that are your own can yield entirely new avenues of exploration.”
Creating small open tools for publishers to build their own unique platforms will amplify our capacities, allow the improvisation and fluid ideation that is necessary in creating new works of digital fiction. Even though software companies urge us toward fully-digital workflows, we are reminded that creative people collaborate well using plain paper. We aspire to carry the spirit of improvisation and brainstorming through the entire creative process and not compartmentalize it within the early pre-digital phase of a project. Improvising throughout a project has the potential of introducing media experimentation along with conceptual ideation.
Media researcher Paul Nemirovsky argues, “in order to facilitate this kind of exploration, (1) computational tools must actively participate in the creative process and (2) the interaction framework must allow structural exploration of media. This leads to our main claim: improvisation should be considered a valid and appropriate paradigm for media interaction.”
We might ask ourselves how might Stein, Woolf, Vonnegut, Burgess, Burroughs or Dahl have approached digital fiction narratives? How would they have found ways to experiment with media and create experiential narrative structures? It is not until we forge our own tools that allow us to engage deeply and fluidly with media, and meaningfully explore creativity and new genres.
Leave a Reply