Nonlinear and Interactive Storytelling
Joshua A. Fisher, Assistant Professor of Emerging Media Design and Development, Center for Emerging Media Design and Development, Ball State University, Muncie, USA
Course Type: Design and Technology-focused
Keywords: Augmented Reality Storytelling, Twine, Graduate, Snapchat, Interactive Nonfiction, Interactive Journalism, Immersive Journalism
University | Department | Level | Credits | Length | Medium |
Ball State University | School of Journalism and Strategic Communication | Graduate | 3 | 15 weeks | In-person |
Course Description
This course explores principles of nonlinear storytelling and non-traditional narrative architectures and experiences. The course introduces students to frameworks for interactive storytelling. Students will learn these basic principles through applied communication design and explore the design process for testing and creating narrative experiences that rely upon user interaction.
Weekly Outline
- Course Introduction
- The History of Interactive Storytelling
- Interactive Storytelling Platforms
- Interactive Narrative Theory
- The Role of Agency in Interactive Digital Narrative
- The Role of Transformation in Interactive Digital Narrative
- The Role of Immersion in Interactive Digital Narrative
- Ludonarrative Harmony and Mechanics in Interactive Storytelling
- Interactive Storytelling in Augmented and Virtual Reality
- Interactive Storytelling in Journalism
- Immersive Journalism Research and Material Gathering
- Augmented Reality User Interface Design and XR Storytelling Structures
- Immersive Journalism Mid-Fidelity Development and Testing
- Immersive Journalism High-Fidelity Development and Testing
- Immersive Journalism Final Usability Evaluations
Course Objectives
- Read and analyze seminal works in nonlinear and interactive storytelling and experience those artifacts.
- Read and analyze theoretical, design, and technical texts describing nonlinear and interactive storytelling properties, technologies, and design approaches.
- Study current technologies for nonlinear and interactive storytelling
- Study current implementations (stories) of nonlinear and interactive storytelling on analog, digital, and immersive (Augmented and Virtual reality) platforms
- Plan and prototype a nonlinear and immersive story for an analog, digital, or immersive (Augmented or Virtual Reality) platform.
Reading
- Research into Interactive Digital Narrative: A Kaleidoscopic View, Janet Murray
- The Garden of Forking Paths, Jorge Luis Borges
- Narrative, Interactivity, Play, and Games: Four Naughty Concepts in Need of Discipline, Eric Zimmerman
- Let the Punishment Fit the Twine: Ancient Greek Punishments as Hypertexts (Web)
- Introduction of Story and Discourse, Seymour Chatman
- Narrative, Media, and Modes in Avatars of Story, Marie-Laure Ryan
- Towards a Specific Theory of Interactive Digital Narrative, Hartmut Koenitz
- Chapter 4, Immersion, Hamlet on the Holodeck, Janet Murray
- Chapter 5, Agency, Hamlet on the Holodeck, Janet Murray
- Chapter 6, Transformation, Hamlet on the Holodeck, Janet Murray
- Transformation Entry, INDCOR Whitepaper 1, INDCOR
- Ludonarrative Dissonance in BioShock, Pat Healy
- Ludonarrative Hermeneutics: A Way Out and the Narrative Paradox , Christian Roth
- Strong Concepts for Mixed Reality Narratives, Joshua A. Fisher
- Challenges of AR Technology, The Design of Virtual and Augmented Reality, Ali Heston
- Journalism that tells the future: Possibilities and journalistic scenarios for augmented reality, Santiago Tejedor-Calvo
- Designing Lived Space-Community Engagement Practices in Rooted AR, Kelsey Cameron and Jessica FitzPatrick
- Twine Cookbook, Dan Cox
IDE and IDN Authoring Tools
- Twine
- Snapchat’s Lens Studio
IDN Artifacts
- Colossal Cave Adventure (Web)
- Hitchhiker’s Guide to the Galaxy (Web)
- Victory Garden by Stuart Moulthrop (Web)
- Sunshine ’69 (Web)
- Figurski at Findhorn on Acid by Richard Holeton (Web)
- ELIZA (Web)
- Every day the same dream by Molleindustria (Web)
- Rainy Day (Web)
- Let the Punishment Fit the Twine (Web)
- Temple of No (Web)
- Life is Strange – Episode 1 (PC/ Mac)
- Your Phone Dies in 20 Seconds (Web)
- Dot’s Home (PC/ Mac)
- Façade (PC)
- Karisvale (PC/ Mac)
- Dys4ia (Web)
- Tale (PC)
- Resonance of the Ocean (PC/ Mac)
- Homeless Realities-Jennifer’s Tent (Web)
- Homeless Realities-“Big Jimi Red” (Web)
- Breonna Taylor App, Lady Phoenix (Web)
- Augmented Reality National Geographic Magazine (YouTube)
Viewings
- Bandersnatch, Black Mirror (Netflix)
- Immersive journalism: using virtual reality to tell the story, Nonny de la Peña, TEDxBerlin (YouTube)
- The Last Chinatown (YouTube)
- Innovations in AR and the Future of Interactive Journalism, Mobile Me & You 2021, Ray Soto (YouTube)
Major Assignments (being assignments whose value is of 25% or more)
Twine Interactive Digital Narrative
- Platform: Twine
- Purpose: This assignment allows students to showcase their IDN design, development, and implementation skills. Further, it helps them practice UX and UI design for browser-based IDNs. Considering the importance of narrative game mechanics, the assignment allows students to explore their design by applying IDN theory. Students develop their Twine IDN over seven weeks. Students iterate on their work each week while implementing theory and practical considerations from the lectures. The project includes a writer’s statement where students reflect on their production process.
- Requirements:
- Project Length: Playthroughs should take 20 to 30 minutes.
- Project Size: At least two endings that are achieved through the effective implementation of dramatic agency.
- Project Aesthetics: Aesthetics for this project must reflect the narrative world’s theme and tone. Students are encouraged to consider visual and sound design in their experiences to increase narrative immersion.
- Coding Proficiency: Students must use Twine Macros, conditionals, variables, and other common data types and structures as part of their production. This work involves learning or using some basic HTML, CSS, and JavaScript proficiency. Instructors are encouraged to produce extra tutorials and videos of various Twine script implementations. In addition, students learn to deploy their experience to Itch.io to embed their projects in their portfolio easily.
- Evaluation:
- Interactivity: To earn a high grade for interactivity, students must demonstrate that they could implement fundamental non-narrative interactions, such as turning the sound off or on, and narrative game mechanics, wherein interactions transparently drove the story. In other words, the students designed choices and interactions wherein the user was unaware they were impacting the story. Further, to teach ludonarrative harmony, students had to include ludic interactions that influenced the progress of their narrative.
- Story and narrative: Students must produce a compelling narrative experience, as validated by instructor and peer reviews. This ideation involves creating and implementing well-developed characters, settings, and dialog. The written material has to be error-free, and its style demonstrates the aesthetic tone and theme of the piece. Accordingly, the work had to have a clear structure, such as those suggested by Marie-Laure Ryan and Janet Murray. The story had to have multiple meaningful endings—dead ends in IDN structures did not count as endings. Additionally, the IDN needs to encourage replay by creating a strong desire in the interactor to replay a decision or experience the narrative through a different character’s perspective. Lastly, students must complete an original story, novel in its perspective, and not a retelling or remix of an existing narrative.
- Production values: Students are encouraged to use visuals and audio in their experience. Students can use Generative AI tools to produce material for this project. The benefit of using Generative AI for their production is that it increases the perceived quality of the final work. That said, instructors need to teach students about ethical approaches to using Generative AI as part of this process. Those lessons must include discussions around training data, giving credit to human creators, and naming the Generative AI tool used in the IDN credits. Students engage in two in-class peer review sessions to achieve a high production value in the work. During these sessions, students are encouraged to help their peers through coding, design, or creative examples and prompts.
Immersive Journalism Project
- Platform: Snapchat’s Lens Studio
- Purpose: Students create an AR journalism experience that uses the affordances of Snapchat’s Augmented and Mixed Reality and what they have learned about IDN and Immersive Journalism design thinking. The final experience the students produce allows an interactor to learn about a nonfiction subject within their local or regional community. In this project, building branching or conditional logic in the experience is not as important as allowing interactors to explore an experience from multiple perspectives. Students have five weeks to produce a mid to high-fidelity prototype. The project enables students to develop their research skills, demonstrate proficiency in AR design and development, and explore emerging affordances for AR storytelling.
- Requirements:
- Project Length: 5 to 10 minutes
- Project Size: The project had to include a cover or welcome screen describing the IDN, an exploration of a topic from at least two perspectives, and a call to action at the end of the experience.
- Project Aesthetics: Students were encouraged to review the provided literature on the latest AR design principles. Beyond those principles, students had to implement an aesthetic that reflected the thematic tone of the journalistic story. However, students must balance aesthetic choices with challenges around the size of the AR experience.
- Coding Proficiency: None. Lens Studio relies on visual scripting they call behaviors to create interactions.
- Evaluation:
- Interactivity: The final experience story needs to effectively use Snap’s affordances to intentionally expand the meaning and potential knowledge that an interaction can learn in the piece. The AR piece needs to be easy to navigate and utilizes UI clues to direct interaction action. Further, The project must effectively use interactive grammars to encourage the interactor to learn about the subject. Lastly, the interactions align with the content and themes of the nonfiction subject.
- Story and narrative: Students must develop the story from multiple sources, potentially interviews and other expert sources. Further, these sources must support claims and add depth to the story. Student research needs to provide in-depth analysis and detail, helping the interactor gain a nuanced understanding of the subject matter. And lastly, the experience needs to present multiple perspectives on the issue, providing a fair and unbiased portrayal of the subject matter.
- Production values: The story’s quality and writing are essential for this project. The script, either through voice-over or in text, needs to be clear, concise, and error-free. The piece needs to be engaging and maintain the interactor’s interest. Further, the writing must effectively convey the information meaningfully, accurately, and compellingly. Critically, the experience must be well-crafted and polished after considering instructor and peer comments. Lastly, the student has taken creative and experimental risks that pushed beyond what the tutorials provided to realize an original and creative experience.
Course Best Practices
- The other 50% of the course is dedicated to technical practicums and reading responses. Technical tutorials begin with step-by-step directions over screen share of coding or design processes. Students then expand upon the lesson to demonstrate proficiency or mastery of the process. For example, students are taught to create 2D UI elements in Lens Studio. As an extension, they make a UI layout comprising buttons with neutral, hover, and clicked states. Similarly, the reading responses require students to synthesize the week’s readings into an analytical lens for an artifact of their choice. For example, reading Narrative, Interactivity, Play, and Game: Four Naughty Concepts in Need of Discipline by Eric Zimmerman and Koenitz’s Towards a Specific Theory of Interactive Digital Narrative and then utilizing the insights from those texts to explore an IDN artifact of the student’s choosing.
- It can be helpful to have teaching assistants or a center where students can get assistance with coding. If these are unavailable, the instructor is encouraged to host an asynchronous message board where students can collaborate on challenges. Additionally, hosting an open hour where multiple students can drop in can be beneficial. This time is not an office hour but an available time where an instructor can pair off students to work with one another on challenges.
- The class met for three-hour studio sessions. In addition to these sessions, the instructor was available for three office hours and one open hour.
- Most of the three-hour sessions were broken down into one hour of lecture and discussion, an hour-long technical tutorial, and then an hour of studio production time. Each large project receives two classes for peer reviews. These reviews may take the entire period depending on the class size.
- Generally, the reading responses in the class are meant to demonstrate an understanding of theory and use it in argumentation, analysis, and ideation. Technical tutorials are meant to build coding and design skills. The major projects allow the students to put theory into practice as they develop an IDN experience.
- The instructor is encouraged to produce video tutorials for the Twine project: click, mouse over, and mouse out macros. Further, technical tutorials should cover video, audio (delays, one-offs, and looping tracks), modals, inputs, singular and multiple conditionals, variables, and iteration. Lastly, the instructor should produce tutorials on CSS and stylesheets for the entire IDN, individual passages, and different elements of the Twine system. Additionally, if students do not have a server for their IDN assets, they can use Flickr or Wix for images and the latter for audio.
- For the Immersive Journalism project, the instructor is encouraged to produce video tutorials covering the following: contextual UI, audio elements, importing 3d models, combining animations, behavior scripting, 360 portals, custom triggers, image tracking, and chaining behaviors to create complex effects.