Audiovisual and Multimedia Languages

Arnau Gifreu-Castells, Ph.D., Audiovisual Communication and Advertising Department, Autonomous University of Barcelona

Jaume Cuspinera, M.s.c., Audiovisual Communication and Advertising Department, Autonomous University of Barcelona

Course Type: Social Science-focused

Keywords: Audiovisual Narrative, Multimedia Narrative, Language, Information Architecture

University Department Level Credits Length Medium
Autonomous University of Barcelona, Spain Audiovisual Communication and Advertising Undergrad 6 15 weeks In-person

Course Description

Theoretical-practical subject that provides an approach to audiovisual and multimedia languages. Based on a set of projects, guidelines are defined for designing and implementing these languages. The theoretical part examines the fundamentals of the audiovisual (narrative, grammar and script) and multimedia (narrative, architecture and design) languages, and analyzes the main trends of current interactive and immersive formats (Interactive Narratives, Interactive Video, Video Games, Transmedia Narratives, Extended Reality/XR, Immersive Experiences, Generative AI, etc.). These trends are projected, navigated and explored based on the study of a selection of works that operate as case studies. The works include both international projects and works produced in the professional and academic environment by the teacher. In the practical part of the course, students work in groups and have specific roles to produce an audiovisual project and a multimedia project that have a thematic and aesthetic relationship. The projects also maintain coherence and a narrative dialogue between the different media and platforms.

Weekly Outline

  1. Presentation of the course: Program / Method / Competences / Evaluation
  2. Topic 1 / Fundamentals of audiovisual language: 1.1 Audiovisual narrative + 1.2. Audiovisual language (technique, shots, composition) 
  3. Topic 1 / Fundamentals of audiovisual language: 1.3. Script (rundown, literary script, technical script, storyboard) + 1.4. Audiovisual rhetoric (tropes and figures) 
  4. Practice 1 AV: Narrative synopsis (with definition of characters, conflict, point of view, emotional strategy, concept and thesis) + Practice 1 MUL: Introduction to the content management system (CMS WordPress)
  5. Topic 2 / Fundamentals of multimedia-interactive language: 2.1. Interactive Narrative (interactive structures) 
  6. Practice 2 AV: Scaletape – script (with study of rhythm, progression, structure, dimension, dialogues and effectiveness of the story) + Practice 2 MUL: Advanced CMS concepts
  7. Topic 2 / Multimedia-interactive language fundamentals: 2.2. Information architecture (sitemap, workflow) + 2.3. Interface design (wireframing, prototyping) 
  8. Practice 3 AV: Rough cut (evaluation of the decisions taken in the script and in the shooting and possible modifications) + Practice 3 MUL: Introduction to Klynt / Connection and publication of social networks.
  9. Topic 3: Expanded – Extended Narratives (XR): 3.1. Augmented Reality + 3.2. Interactive narratives + 3.3. Interactive video
  10.  Group tutorials (groups A + B)
  11.  Topic 3: Expanded – Extended Narratives (XR): 3.4. Video games + 3.5. Transmedia Narratives + 3.6. Immersive experiences / Virtual Reality + 3.7. Generative AI + Part B group presentations (script software + architecture 15 min. per group approx.)
  12. Group tutorials (groups A + B)
  13. Group tutorials (groups A + B)
  14. Exhaustive review of exam theory + Part B group presentations (script software + architecture 15 min. per group approx.)
  15. Exam

Course Objectives ​

  • To provide the general concepts and theoretical bases of audiovisual language as an effective tool for producing messages and contents.
  • To provide basic knowledge about digital media languages, with particular emphasis on their technological and cultural specificities.
  • To gain a deeper understanding of the techniques and specific knowledge necessary for creating contents in digital media.
  • To acquire a better knowledge of the technological aspects related to production, publication and dissemination of digital contents.

Reading

  • El lenguaje de los nuevos medios de comunicación. La imagen en la era digital – Lev Manovich (Paidós, 2005)
  • El viaje del escritor: Las estructuras míticas para escritores, guionistas, dramaturgos y novelistas – Christopher Vogler (Ediciones Robinbook, 2002)
  • Expanded Cinema – Gene Youngblood (Studio Vista Limited, 1970)
  • Hamlet on the Holodeck: The Future of Narrative in Cyberspace – Janet Murray (MIT Press, 1998)
  • Las leyes de la interfaz: Diseño, ecología, evolución, tecnología – Carlos Alberto Scolari (Editorial Gedisa, 2018)
  • Lector in Fabula – Umberto Eco (Bompiani, 1979)
  • Narrative across media. The languages of storytelling – Marie Laurie Ryan (University of Nebraska Press, 2005)
  • Pause and Effect. The art of interactive narrative – Mark Stephen Meadows (New Riders, 2003)
  • Remediation. Understanding New Media – David Bolter & Richard Grusin (MIT Press, 2000)
  • The Art of Interactive Design – Chris Crawford (No Starch Press, 2002)
  • The Hero with a Thousand Faces. Understanding the Hero’s Journey Through Psychology and Mythology – Joseph Campbell (The Collected Works of Joseph Campbell series, 1948)
  • The New Media Reader – Nick Monfort & Noah Wardrip-Fruin (MIT Press, 2003)
  • Writing for Multimedia. A Guide and Sourcebook for the Digital Writer – Michael D. Korolenko (Wadsworth Publishing Company, 2001)

Viewings

IDE and IDN Authoring Tools

  • Word & Powerpoint (Office Tools)
  • Da Vinci / Premiere / Final Cut / Affinity Photo / Photoshop / Audacity / Audition (Video & Audio Editing Tools)
  • Miro (Brainstorming Tool)
  • Figma (Information Architecture Tool)
  • WordPress (CMS Tool)
  • Klynt – HTML – HP5 plugin (WP) (Multimedia Editor Tools)

IDN Artifacts

  • Bugarach. Cómo sobrevivir al apocalipsis [Transmedia Documentary] – Sergi Cameron (Nanouk Films; RTVE.ES; Interactius; Playfilm, 2016). http://lab.rtve.es/webdocs/bugarach/
  • COME/IN/DOC [Collaborative Meta Interactive Documentary] [Interactive Documentary] – Arnau Gifreu (MIT Open Documentary Lab; interDoc; i-Docs; interDocs Barcelona, 2016). http://comeindoc.com/
  • Cyborg Project [Transmedia Documentary] – Roger Soldevila (Espill Media, 2014). http://cyborgproject.com
  • El Cubo. Historias tridimensionales [Interactive Theater/Fiction/Documentary] – Juan Baquero (RTVCPlay; Interfaz, 2021-2023). https://elcubo.rtvcplay.co/
  • Escape Europe. A Webdoc Game on the Southern Border [Immersive Documentary] – Oscar Martinez & Domingo Sánchez-Mesa (ASAD-Asociación Solidaria Andaluza de Desarrollo. Universidad de Granada. ARCI Lecce. Università del Salento. University of the Aegean. European Union, 2022). https://utopi.es/escape-europe/
  • Escarabajos. Un país descubierto a pedalazos [Transmedia Documentary] – Zuly Zabala (Red de Bibliotecas. Banco de la República, 2021). https://www.banrepcultural.org/exposiciones/escarabajos
  • Interactivos La Marató TV3 [Informative interactives – documentaries] – Catalina Acelas, Pere Arcas & Arnau Gifreu (Espill media; Ask Yourself; Televisió de Catalunya; Fundació La Marató de TV3, 2013-2014). http://agifreu.com/marato2013/ / http://www.agifreu.com/marato2014/
  • Las Sinsombrero [Transmedia Documentary] – Tània Balló, Manuel Jiménez & SerranaTorres (Intropia Media; Yolaperdono; RTVE.ES, 2015). http://www.rtve.es/lassinsombrero/es/webdoc/
  • More than Bodies [Transmedia Essay] – Núria Fontané (ERAM – University of Girona, 2017). http://morethanbodies.com/
  • Pregoneros de Medellín [Transmedia Documentary] – Angela Carabalí & Thibault Durand (Grupo Carabalí. MINTIC. Vive Digital, Baobab, 2015). http://pregonerosdemedellin.com
  • Sexo, maracas y chihuahuas [Transmedia Documentary] – Diego Mas Trelles & Albert Solé (Minimal Films; Televisió de Catalunya; RTVE.ES; Labyrinz Storymaking Studio, 2016). http://lab.rtve.es/webdocs/xavier-cugat/#/

Major Assignments (being assignments whose value is of 25% or more)

Interactive Project

  • Platform: Web/Interactive
  • Purpose: elaboration of the different stages to produce an interactive piece
  • Requirements:
    • Project Length: free choice
    • Project Size: free choice
    • Project Aesthetics: type of interactive structure, user experience design, interface design, usability analysis
    • Coding Proficiency: use an appropriate architecture and prototyping software (Figma), use of design tools (Photoshop, Indesign, Illustrator, etc.), use of web development languages (HTML, CSS, Javascript, React, MySQL, PHP, etc.)
  • Evaluation:  The evaluation of the course will be based on 3 axes: theory (50%), practical exercises (40%) and seminars (10%). To pass the course it is necessary to pass all the axes independently.
    1.  Theory (50%) = multiple choice exam / 10 written questions.
    2. Practice assignments (40%) = delivery of 6 practice assignments (3 audiovisual and 3 multimedia)
    3.  Seminars (10%) = attendance and participation.
    • Interactivity: Developing the interactivity must include the elaboration of a comprehensive and functional information architecture consisting of presenting deliverable documents (navigation map, flowchart, etc.) and wireframes. The software recommended is Figma. It can be used effectively to design interactivity in the strong sense of the term (interactive structures), i.e. the audiovisual is at the service of interactivity and not the other way around (a native interactive format, interactive short, documentary, series, essay, etc.), and has a higher value than a web presentation and advertising of an audiovisual product (creating a simple website using a CMS like WP).

    • Story and narrative: In addition to evaluating the final piece (story, composition, rhythm, editing, etc.), we also assess and give value to an impeccable execution of the narrative synopsis (including characters, conflict, point of view, emotional strategy, concept and thesis), scaletape – script (with a study of rhythm, progression, structure, dimension, dialogues and effectiveness of the story) as well as the rough cut (evaluation of the decisions taken in the script and shooting as well as possible modifications).

    • Production values: The production plan and the type of planning carried out in general is evaluated. In this sense, students must deliver an exhaustive chronogram of both the audiovisual and multimedia assignments. Delivery within the deadlines set for each delivery phase is also evaluated.

Audiovisual Project

  • Platform: Audiovisual (TV, Internet, OTT)
  • Purpose: elaboration of the different stages to produce an audiovisual piece
  • Requirements:
    • Project Length: 5-10 min
    • Project Size: free choice
    • Project Aesthetics: composition, rhythm, editing
    • Coding Proficiency: recording in full HD (1920×1080) or 2K/4K, editing in Final Cut/Premiere/Davinci, possible use of post-production programs such as After Effects, etc.

Course Best Practices

  • Short theory blocks complemented with a showcase of national and international examples.
  • Online guided navigation of projects that the teacher has produced in different environments: academia, festivals, freelance and TV-platform streaming.
  • Tutorials in the audiovisual and multimedia development phases given by 2 specialist teachers.
  • Interactive exercises/games with students to choose the type of project and development phase they want to prioritize.
  • Short presentations of the analysis of audiovisual and multimedia scripting programs, optional as extra subject points.
  • The course takes place once a week (on Mondays) for 3 hours with a 15-minute break, both for theory and practice. The dynamics vary depending on whether it is a theoretical or practical class.
  • Sessions are alternated between reflection and discussion as a result of the theoretical approach, practice and project follow-up tutorials.
  • The course provides three types of resources as an additional complement to the classes: first, a series of readings and reviews based on books, articles and posts. Secondly, a set of optional viewing of audiovisual films and documentaries; and thirdly, a selection of educational projects that provide a deeper understanding of several of the key topics of the course (Moments of innovation, COME/IN/DOC, Biology of Story, etc.).
  • The practice assignments focus on and are synchronized with each block in relation to the audiovisual and multimedia narrative. In this sense, learning is gradual and as the theory is explained and the materials are analyzed, the different assignments are handed in.
  • The works are screened at the end of the course in front of the class (1); Social networks are used to disseminate and spread the projects (2); The audiovisual content is exhibited on video aggregator platforms (3); Projects are presented at university sections of festivals (Film, Documentary, Multimedia, etc.) (4); and Projects are presented at the ‘University Pitching’ organized by the Audiovisual Cluster of Catalonia (5).