Ethereal Overlay Final Slides

14 April, 2014

IMD 3109 - Design Studio 3 James Acres

Ethereal Overlay Angel Abundo Justin Loranger-Ahluwalia Michael Hardie Dylan Wallace

1. Introductions

1.1 The Team Angel Abundo

Michael Hardie

- Art & UI lead - Worked on video capture

- Audio lead - Worked on texture art - Researched collision technology

Justin LorangerAhluwalia

Dylan Wallace

- Game design & documentation lead - Worked on art & video capture

- Programming lead

1.2 The Project

Ethereal Overlay is an augmented reality game platform aimed at turning any surface into a video game screen. Using surface detection & 2D projection mapping technology, walls, drawings, & structures can become fully functional gameplay elements.

Technologies Used

Playstation Eye Camera, Openframeworks, OSC, OpenCV, Cubase, Audacity, Adobe Audition, Processing, Android Smart Phones, Projector

2. Project Demonstration

3. Behind The Scenes

3.1 Implementation Process Projection Calibration - A calibration image is projected onto the surface - The camera takes a snapshot of the surface, & uses the projection to set the screen parameters - Camera & game screen data is re-cast to suit projection space

Image Reading & Setting - Camera Feed is read with OpenCV & allocated to a variable - A threshold function is performed to distinguish colours & set them to binary maps (right) - Coloured thresholds are combined into one image, which is sent to game

3.1 Implementation Process Collision Detection - Map generated by placing gameplay layers over collision map, based on the pixel colours of the collision map - Collision handled by inverse acceleration

Network Communication - Phone communication done by sending a message with address from phone - Connects to local network via IP - Uses OSC for communication

Audio Creation - Generate a melody with a base audio (Cubase, Audacity) - Enhance the base instrument & add secondary instruments - Mix the sound (Audition)

3.2 Challenges Scrapped Research - Researched several potential implementations, some of which were dropped - Box2D (2D physics engine for collision) was dropped because it required box shapes; more dynamic shapes would require complex edge detection - Openframeworks was tested for Android phone integration, but was extremely strict about installation setup & environment requirements

Openframeworks & Library Setup - Setting up the game libraries required specific setup, including installing Miscrosoft Visual Studio 2012 for debugging - This setup was time-intensive, & resulted to technical issues separate from code

3.2 Challenges Image Reading & Storage - Image processing required several passes & workarounds due to an overabundance of libraries & methods - Data retrieval method used OpenCV to get camera data (ofxCvColorImage), while the thresholding method used Openframeworks functions (which required ofImage) - Image data type needed to be converted & workarounds used to work properly

Phone Communication - Plans for project included phone app controls (right) - Phone would send Processing app through a local network to communicate with game - Permission for sending & receiving could not be achieved, so game could not receive app information - Problem pinointed to internet setting on phone

3.3 Expansion Plans Greater Game Customisation - Character customisation & alternate ship designs - Human player characters - Player accounts - Scoring & leaderboards - AI player option

3.3 Expansion Plans Greater Game Depth - Themes (right) - Animated scenery/obstacles - Terrain penalties - Weapon upgrades (below)

3.3 Expansion Plans More Games & Game Modes - Team Deathmatch, Survival, Capture the Flag, etc.

- Racing game - Platforming game

More Hardware Depth - Beyond the whiteboard (reading complex surfaces) - Thresholding with textures & materials - Obstacles: object method vs pixel method - Dynamic lighting - Online multiplayer

4. Questions & Answers