Concept – Headphone Maps 2014

The Lost Souls of George Square is a fun and light hearted mobile game based around the real world location of George Square in Edinburgh. Inspired by classic arcade and retro gameplay, the player takes on the role of a well meaning ghost hunter, who must free animal’s souls which have been cursed by an evil witch. To save the ghosts the player must navigate towards locations mapped onto the physical space of the square, and when they are close enough, shake their phone all around until the ghosts are free. Four souls are trapped in the square, and on saving all four, the player will have completed the game.

The game features reactive binaural audio which alerts the player to the ghost’s locations. A clean and simple visual display gives the user information relating to the distance and location of the ghosts and the player’s current progress through the game. GPS sensors, magnetometer, and accelerometer data are used to track the player’s location and direct them towards fixed locations in physical space.

The game concept references well known games such as Nintendo’s Luigi’s Mansion where a character carries a hoover device and captures ghosts from different locations. The keywords that may be used to describe the whole experience can be – fun, entertaining, quick, humorous, silly, paranormal, retro, arcade, eerie and amusing.

Our priority in the project was to focus on simple systems that when combined lead to immersive and emergent gameplay. By basing the game in a well known physical space, the player is able to explore and see this environment in a new way. George Square was selected as the ideal location as its a small location with a mix of open and closed areas filled with trees. This kind of a location also seemed to fit the narrative as there are lots of eerie and interesting artefacts in the area (labrynth, statues, etc).

 

Designing the Visuals

With visuals playing an important role within the project, not just for providing feedback to the player, but also to give our project a visual presence, it was important to have visuals that were appropriate and fit for purpose for what we wanted to do. The majority of the visuals were produced to work fittingly with the music and sounds accordingly. With initial plans in the beginning of having a more darker and scarier visual style (to go along with a ghost hunting/finding game), the visuals were going to be created within Adobe Illustrator so as the assets for our project would be in a vector format, which would allow us to scale the images accordingly depending on what size of graphics we needed for a specific location on the screen, or task. Continue reading

Designing the Sound

Getting the sound right was crucial as it was one of the most important cues the user had to refer to. Given below are some of the most important sonic designs used in the game.

Voiceovers
Instead of using sound effects, we decided to personify each of the animal souls through humorous dialogues. This made the whole experience a lot more entertaining and the funny liners are one of the highlights of the whole experience. All the voiceovers for the game were done by Daniel McGurty. A pitch shift and chorus effect were used to give the voiceovers a paranormal character. A sample of some of the lines can be heard below.

Transition Effects
In order to transition between different states (from the search state to the shake state), we used a combination of thunder and lightning sounds, after which the scary themes play. As the player gets closer to the vicinity of the soul, there are beeps added to match the visual meter and also a chorus effect, which increases intensity and depth as the player gets closer.

Sonic Tutorial
The tutorial is the first thing the player hears in the game. Hence, it had to be well designed and dramatic in order to set a good first impression. The first part of the tutorial explains the context and narrative, and the second half explains the rules and gameplay. A lot of binaural effects have been used to match the theme of the game and it sets the mood for the rest of the experience. The voiceover was done by Daniel McGurty.

Ending Theme
This sound plays at the end of the game to signify that all the souls have been rescued.

Designing the Music

For this project, the music was designed with these intentions:

  1. The music should hold all the elements together and be part of the game’s ‘character’.
  2. The music should also be part of the cues that depict the state and progress of the game.
  3. The music should set the mood of the whole game.

The first challenge was creating the main theme for the game which would serve as the main reference point for the rest of the themes. Since the overall theme of the game was meant to replicate an arcade console game (8-bit like), the music needed to fit well into the theme. The soundtracks of Sonic the Hedgehog, Mario Bros, Luigi’s Mansion and other popular SEGA/Nintendo games were used as the main references. In order to make it sound ‘entertaining’ and ‘fun’, the main tune was composed in the style of Ragtime (mostly in the style of Scott Joplin).

Initially, the main theme was supposed to play during the ‘search phase’ at all levels (when the player is looking for the souls) and the other ‘scary’ themes were triggered when the player was close to the vicinity of the soul. Below are snippets from the ‘scary’ themes, which were designed similarly to the main theme.

After the final presentation, it was decided to include one more theme for the ‘search phase’ of the game which would be more sparse and eerie. To give it a spookier effect, extra sounds and ambiences were added to the loop.

All of the above themes are loopable. A switch from one theme to another happens with the aid of a thunder sound (which acts as a transitioning element, as well as a cue that the player has reached the vicinity of the soul).

Considering the fact that these themes may not at all give out a ‘spooky’ or ‘horror’ feel (which is usually the vibe associated with anything paranormal), it was designed with the main intention of making the whole experience sound ‘fun’. It was a indeed a challenge to make the music fit into context. The feedback from the presentation was mostly mixed regarding the music; most of the users felt that the music fit the theme but some of them felt it was a distraction and could have been avoided altogether. Overall, it was felt the music still fills in a lot of silent gaps in the game, and since its continuous, it also gives the impression that the game is in progress.

 Tools used to build the Music
DAW – Logic Pro X (www.apple.com/logic-pro/)
8 Bit Sounds – c700 (picopicose.com/software.html)

Technical Documentation

Project outline:

From a technical standpoint, the aim of this project was to create a location aware binaural audio game running natively on Android. The main design considerations were to create a system which gave clear and consistent user feedback, responded consistently to the user’s input, and was easy to use and learn. The application was to feature dynamic audio and visuals which responded to the direct input of the player and their physical location. Key to the success of the code was for this information to be portrayed as simply as possible in order to avoid confusion and aid learning.

Below I have given a descriptive account of the application, giving a overall structure of the code, followed by a more detailed breakdown of the processes involved in each stage. I also give comments regarding technical problems that arose during the test session, the current state of the code, and future solutions.

For a more technical understanding of the application, the fully commented code should be downloaded from: adamcampbell.org/dmsp/TheLostSoulsOfGeorgeSquare.zip

The source Java file is located at
/src/com/example/thelostsoulsofgeorgesquare/LostSoulsMain.java

The Pure Data patch is contained in
/res/raw/loustsouls.zip

‘Finding’ state refers to the player searching for the ghost, before they have crossed the distance threshold, ‘battle’ state is when user has crossed the threshold and is interacting with the ghost.

Sensors Used

GPS sensor – Used to measure the player’s current location and calculate their bearing to and distance from pre-set dummy locations.

Magnetometer – Used to calculate player’s orientation to dummy ghost locations.

Accelerometer – Used to calculate shake behaviour.

Software used

Pure Data & LibPD – Pure Data is used for all audio components and some game logic. Embedded into application with LibPD.

Processing – Used for all visual components

Java and Eclipse – Used for collecting sensor data and the majority of game logic.

External libraries / code – ProcessingTouchEvents and UiImageButton (twobigears.com/), tbe_3DCeption~ external for Pure Data (twobigears.com/)

Overall Programme Structure

– read sensor data
– calculate user’s location
– calculate distance of user to next game location
– calculate user’s bearing to game location
– if user is within 10 meters of ghost GPS location, trigger Battle state and allow shake behaviour
– calculate overall magnitude of three poles of accelerometer
– if magnitude is over threshold trigger victory sound and display victory screen
– if user has completed less than 4 stages direct player to next GPS location
– if user has completed 4 stages trigger final victory sound and end screen.

Detailed breakdown of each section

Intro
Visuals
– Static visual screen.
Audio
– Pre-recorded introduction sequence is played.
Logic
– Hold image on screen until sound file has finished, then trigger next game state.

Finding state
Visuals
– Compass which rotates and points towards next ghost location relative to user’s current orientation (low pass filter applied to smooth out data).
– Counter of the number of ghosts saved and the current game level.
– Reset button to restart the game.
– Distance meter which fills as the player gets closer. This moves in stages rather than in a smooth movement to counteract the slow and unpredictable rate at which GPS data updates.
Audio
– Ghost voice, which is triggered randomly (frequency and phrase) and is panned binaurally towards ghost location, using azimuth.
– Music is a loop with added chorus effect which increases as the user gets closer to the ghost location, adding a warbling / out of tune effect.
Logic
– User’s GPS location measured by phone.
– Distance from user’s location and pre-set ghost GPS location is calculated
– User’s bearing to ghost GPS location is measured.
– If user is within distance threshold then trigger thunder sound and next state.

Battle state
Visuals
– ‘Shake’ text instructing the user to shake the phone
– ‘Magic symbol’ image to indicate user can interact with ghost
– Ghost image
– Reset button
– Background colour is linked to the magnitude of the accelerometer and changes as the user shakes the phone harder.
Audio
– When state is first triggered, play thunder sound.
– Ghost voice, triggered more frequently in this stage. Binaural panning is random and moves around the players location to give the impression of the ghost moving around the players head.
– A short music loop which is different for each stage.
– ‘Shake’ audio, linked to the phone magnitude. Increases in intensity and modulation as the player shakes the phone harder.
Logic
– Calculate overall magnitude of accelerometer (low pass filter applied to smooth out sensor data).
– If magnitude passes threshold, trigger Victory state

Victory
Visuals
– Static screen, displays an image of whichever animal has been saved (i.e. level 1 owl, level 2 chipmunk etc.)
– Reset button
Audio
– Pre-composed sound file is played
Logic
– Hold image on screen until sound file has finished, then trigger next state.

End Screen
Visuals
– Static screen image
– Reset button
Audio
– Pre-composed victory sound
Logic
– Hold image on screen until user presses ‘reset’ button to start game again

Comments and possible improvements

During the test session a bug was discovered which allowed the user to advance the game while not in ‘battle’ mode. This lead to user’s becoming confused as the cues did not clearly communicate the game logic. This bug has since been fixed and all planned interactions and behaviours work consistently.

The main issue with the current build is that game logic is shared between Pure Data and Java. This came about due to the manner in which the project was built up, with early prototypes being constructed in Pure Data, but caused difficulties with debugging and editing as the size of the project increased. As a result the code became more difficult to follow, as messages are passed back and forth between the two environments. If continuing with the project I would re-write the code to be as fully implemented in Java as possible, using Pure Data to generate and modulate sounds, but not handle game logic.

This and other style issues in the code (mainly present in the pd patches) are due to a lack of thorough planning before beginning building the project, although don’t affect the running of the application.

Timeline of Events

Here is a timeline of events that led to the final result. An interactive version of this timeline can be found on this webpage.

  • 30th Jan ’14 – Group Meeting 1 – Discussed submissions, concepts and early ideas.
  • 5th Feb ’14 – Group Meeting 2 – Discussed design themes, techical considerations, narratives, themes and ideas – Click Here for Minutes
  • 12th Feb ’14 – Group Meeting 3 – Decided to develop light hearted game in the style of Luigi’s Mansion, Discussed Design, Themes and Interactivity – Click Here for Minutes
  • 19th Feb ’14 – Group Meeting 4 – Planned 1st Submission – Accellerometer Patch Works, Main Music Theme Ready – Click Here for Minutes
  • 26th Feb ’14 – Group Meeting 5 – Submission 1 Ready – Ideal Gameplay Set, Visual Ideas Drafted and Game Logic Works – Click Here for Submission 1
  • 6th March ’14 – Group Meeting 6 – Discussed Game Levels, Gameplay and Design
  • 12th March ’14 – Group Meeting 7 (Level 1 Test) – Tested one level of the game – Patches for Music and Game Logic Ready and Working – Click Here for Blog Post
  • 19th March ’14 – Group Meeting 8 – Tested game with two levels – Progress was good, Music for all levels ready
  • 28th March ’14 – Group Meeting 9 – Tested Whole Gameplay – Tutorial for game ready and all sound effects recorded, visual elements ready but needed to be implemented. Name Decided – Lost Souls of George Square
  • 1st April ’14 – Group Meeting 10 – Whole gameplay ready with Visuals – needed to implement reset button and improve transition between game states, added final sound at the end
  • 3rd April ’14 – Group Meeting 11 & Final Presentation – Final edits to Logic and gameplay – Feedback Received after Final Presentation – Click Here for Feedback Video
  • 9th April ’14 – Group Meeting 12 – Decided to make further edits to music and planned documentation
  • 17th April ’14 – Group Meeting 13 – Planned Documentation, Recorded Gameplay Video and did final tests to gameplay – Click Here for Gameplay Video

Feedback and Improvements

Our final presentation was conducted on 3rd April 2014. The feedback we received from the participants is featured in the video below.

Things that we could have developed if we had more time.

  1. Work on improving the binaural spatialisation.
  2. Have a time based point system to make it challenging and competitive.
  3. Increased the number of levels to a larger area with more distractions at later levels to make it distracting.
  4. Have multiple souls at once time if possible.
  5. Work with random location mapper for souls.
  6. Have more feedback on the mobile device (using vibrations, visuals etc)