Storytelling with XR

by Ajibade Quwiyu, African XR Community, Lagos State, Nigeria

Course Type: Design and Technology-focused

Keywords: Extended Reality, Augmented Reality, Game Programming

University	Department	Level	Credits	Length	Medium
African XR Community	Extended Reality (XR, VR, AR)	Undergrad/Graduate	No Unit	15 weeks	Hybrid

Course Description

This course, grounded in the experiences from the African XR Community, guides students through the creation and optimization of Extended Reality (XR) experiences using Unity. Spanning 15 weeks, participants from varied backgrounds are introduced to the fundamentals of XR, asset creation, and programming, culminating in a capstone project that consolidates their learning in a tangible, portfolio-ready piece.

Course Objectives

Understand the basics and importance of XR in today’s technological landscape.
Create interactive games and experiences using Unity.
Develop proficiency in asset creation and optimization for Unity.
Navigate the complexities of XR project development, from conception to execution.
Adapt learning approaches to accommodate diverse student backgrounds and learning speeds.
Encourage student engagement through rigorous assignment and project completion standards.
Foster a deep understanding of Unity’s C# scripting to create immersive experiences.

Weekly Outline

Week 1.

Introduction to Extended Reality (XR).
Basics requirement for XR.
Installation & Setup of required Software for XR.

Week 2.

Introduction to Autodesk Maya for Asset creation.
Basics Modelling of Asset using Autodesk Maya.
Optimization Techniques.
Weekly Assessment on 3D Modelling.

Week 3.

Introduction to UV Unwrapping.
Foundations of PBR Texturing with UV Unwrapping.
Advance UV Unwrapping.
Weekly Assessment on Modelling and UV Unwrapping

Week 4

Introduction to Texturing and Materials Creation Using Substance Painter and Photoshop.
Exporting of UV and Assets into Substance Painter for texturing.
Alpha Blending, Exporting Textures, and creating GLTF/GLB files in substance painter.
Weekly Assessment on Modelling, UV Unwrapping, and Texturing.

Week 5

Introduction to Unity Interface.
Creating Materials and Environmental staging.
Unity Quick Tips.
Weekly Assessment on Environmental staging.

Week 6

Introduction to Unity C# Programming.
Variables, If-Then Logic, and Loops.
Arrays, Switch Statements, classes, Functions, and methods.
Weekly Assessment on Unity C# Programming.

Week 7

Unity Scripting, Physics, and Movement.
Basic Game Programming.
Basic Game Programming Contd.
Weekly Assessment on Unity C# Programming.

Week 8 -10

Introduction to Augmented Reality.
Using of AR Foundation to Create an AR Experience.
Plane Tracking, Image Tracking, Transparent Recieve Shadows, and Light Estimation.
Build and Fly a Real-Time Physics Driven Drone in Augmented Reality.
Brief Introduction to Fast Prototyping Softwares for Augmented Reality.
SwiftXR, SparkAR, Lens Studio.
Weekly Assessment on Augmented Reality.

Week 11 & 13

Introduction to Virtual Reality.
VR Environmental Staging and Optimization, Avatar Creation.
Teleportation, Grabbing in VR, and UI Interactions in VR.
Weekly Assessment on Augmented Reality.

Week 14 – 24

Personal Projects.
Resume and Portfolio Creation.

Viewings

SocialVerse: Beginner’s Guide, Intro to Social AR: https://youtu.be/YmejxFqiWAA?si=Ni_WvxRuvqbSi_LK
How to Create Stunning 3D Models with Model Viewer’s Augmented Reality: https://youtu.be/x0xS_Gl5W3k?si=ZVQxayrRN2uCz2Qh
Building augmented reality configurators in model viewer: https://youtu.be/V5m8S6G_CUo?si=T3Ot0RKHoIC60yqy
Unity Game Development for Beginners: An Introduction to Creating Your First Game: https://youtu.be/qDM1ZedF1Mo?si=5quZpgMNslen6xts
Unity for Beginners: How to Create Materials and Stage Environments for Your Games: https://youtu.be/hNlimg41HHA?si=2vM9twwsykeQfSzS
Unity C# Programming for Beginners: An Introduction to Game Development: https://youtu.be/uVsIluibLOo?si=Im-pXwiG22HTTjlh
Learn C# Arrays, Switch Statements, Classes, and Functions in Unity: https://youtu.be/nSrrt0Ps-fI?si=Js8XgQHXQUtRQ2KK
Introduction to scripting in unity: https://youtu.be/rwTGpNEd_ro?si=GSYnDCWoFs1x4hA3
Scripting in Unity Cont’d: https://youtu.be/Xt6Fs-GzuBk?si=dOs2cQAVGg7Zl80b
Android Game Development in Unity Engine: https://youtu.be/4zDJ6OXaTmI?si=COb8gneboLrgmWaw
How to Create an Endless Runner Game for Android Devices using Unity Engine: https://youtu.be/UyP8XEBxutU?si=97tY1sg-TjmbNAEO
AR Development for Beginners: Learn the Basics of Augmented Reality: https://youtu.be/3YjBI96Gwaw?si=ywGx7sWH3l4Yfi1G

Reading

“Unity for Beginners” by Unity Technologies
“Introduction to Game Development” by Michael Hartlef

IDN Artifacts

XR Project Showcase (African XR Community Platform)

IDE and IDN Authoring Tools

Unity (IDE / IDN Authoring Tool)

Major Assignments (being assignments whose value is of 25% or more)

Virtual Machine Workshop

- Platform: Unity for Mobile AR and VR
- Purpose: To develop a mobile AR and VR application that facilitates a shared learning experience between instructors and students. This project aims to demonstrate how XR technology can be utilized in educational settings to enhance the understanding of complex concepts through interactive simulations.

Requirements:

Project Length: 10-15 hours of development time
Project Size: Design and implement an XR experience that allows multiple users to interact within the same virtual environment, using mobile devices or VR headsets.
Project Aesthetics: Focus on clarity and usability in the design, ensuring that virtual machines are accurately represented and easy to interact with.
Coding Proficiency: Use Unity C# scripting to enable real-time multiplayer interactions, with emphasis on networked VR/AR experiences using ARCore or similar technologies.

Evaluation:

- Interactivity: Evaluate the ease with which users can join and participate in the shared XR experience. The application should allow for seamless interaction between the instructor’s demonstrations and the students’ experiences.
- Story and Narrative: While a strong narrative may not be the primary focus, the flow of the educational content should be logical and engaging, guiding users through the learning objectives effectively.
- Production Values: High-quality visual assets and smooth performance on mobile devices are critical. The application should maintain a high frame rate and offer intuitive UI/UX for an educational context.

Assignment Video Link

https://www.youtube.com/playlist?list=PLoFsvU11P5r6aBrilIXmkWX8kquJyeNxR

Construction VR

Platform: Unity with Oculus Integration SDK
Purpose: To create a virtual reality application aimed at training construction site workers on health and safety procedures. This project simulates real-world construction environments, enabling trainees to practice and master safety protocols in a risk-free setting.
Requirements:

Project Length: 15-20 hours of development time
Project Size: Develop an immersive VR experience that transports users to a realistic construction site, complete with various machinery and safety scenarios.
Project Aesthetics: Utilize realistic 3D models and environments to mimic actual construction sites. Attention to detail in the simulation of construction equipment and safety hazards is crucial.
Coding Proficiency: Advanced scripting for VR interactions, including the use of PPE, machinery operation, and hazard recognition. Implement scenario-based training with feedback and scoring systems.

Evaluation:

- Interactivity: Assess the application’s effectiveness in engaging users with interactive safety training scenarios. The VR experience should include hands-on tasks that require users to apply what they have learned.
- Story and Narrative: Create a compelling training pathway that leads users through different stages of safety training, from PPE usage to operating machinery, with each stage building on the last.
- Production Values: The VR environment should be highly detailed and accurate to real construction sites. Quality audio design, realistic physics, and user-friendly controls are key for a successful training tool.