Come experiment with us!

sound equipment_MG_3073 copy

 

Brain Drain is an ongoing postgraduate students’ project at ECA.

We are happy to invite you to experiment with us!

Design and Digital Media Msc students are working together with Sound Design and Acoustics Msc students towards the creation of an interactive arts installation, under the supervision of Professor Richard Coyne and PhD student Dorothea Kalogianni.

 

 

—————————————————————————————————-

The innovation of the installation is that it uses a brain interface with the help of portable EEG (electroencephalography).

We are running some experiments that will help us understand how to improve our installation and the user experience. And we would be very happy if you came over and experimented with our installation!

The experimentation time will only take 20 to 30 minutes! And we will offer you a cup of coffee as well as some sweet treat 🙂

Here is the link of a brief introduction video of our porject:
Video: www.youtube.com/watch?v=dWghvaV_RFg

The experiments will take place on Friday the 14th, Saturday the 15th, Friday the 21st and Saturday the 22nd of March.

On Fridays the experiments will take place from 14:00 to 17:30 and on Saturdays from 13:00 to 16:30. Below is a doodle link that you can use to book your desired slot. This way there will be no undesired waiting times.

doodle.com/ph7r6h6xdr37r4qa

ps: We would like you to wear an absolutely safe and easily worn portable EEG headset (emotiv.com/)

Looking forward to seeing you!

 

Brain as a language in the Future

Introduction

After the Tower of Babel
One of the most aspiring communication approach without words and languages is a message for extraterrestrial life. In 1977, NASA launched Voyager Golden Record which are symbolic messages to the space. This enigmatic disk contains more than 100 symbolic pictures of human being, animals, human anatomy, multilingual greetings and natural sounds. As human being did not know the communication style of addressee, they prepared non-literature messages. On the other hand, most part of communication is composed of languages and words. People uses various types of communication tools such as Facebook chat and Skype video message. As for these communicational messages, there is an significant limiting condition, language barrier. However, there are various different types of communication using different sensory such as odor and temperature (Classen, 2005). This installation idea focuses on wordless communication among people in cerebral and synesthestic way.

Concepts of the installation
The narrative of the installation is based on three concepts. The fist one is “brain as a language”, which means that human brains will be able to communicate with each other without words in the future. For instance, some science fiction novelists such as Sir Arthur Charles Clark, Robert Anson Heinlein and Stanisław Lem have proposed characteristics of future human being or extraterrestrial lives. They can share their emotions and thoughts without uttering. Second concept is “coding and decoding in non-oral communication”. For communication between brains, it is significant to apply the way of coding and decoding in these communication. The last idea is “synesthesia”. Brains behaves as a switch of different senses. As invisible feedback from brains have no receiver unlike five senses, the installation needs to convert them into cognitive stimuli.

This one way communication transfers messages from previous participants to next participants as long as there are audience. It would be similar to a telephone game. Some message would convey same emotions, and others would evoke totally different emotions. Brains continues to transfer messages to other brains with changing their messages.

Step 1 : Searching for stimuli
The first exploration starts at searching for stimuli which evoke shareable emotions or meanings. For instance, scratch sounds on the blackboard tend to irritate people. Also, it is seemed that bell sounds in temples make them meditate. All input and output is saved, and clarified relationship between specific external stimuli and emotions. To achieve this purpose, research of non oral communication might be useful.

Communication without language
In the human history, people have tried to communicate with each other without languages. Morse code represents alphabet using just two sounds. Also, lighthouses have communicate with ships lights. Aboriginal Australian have “song lines” or ” dreaming track” which are routes relate to creators of creatures. Song lines is a system to travel across severe Australian wild land without risk of lost. Songs which describes landmarks, dangers and characteristics of each path have been passed down by word of mouth. Sound pitch is one of the most significant point in song lines because some specific pitch evoke scary or frustrate feelings. It is hard for strangers to understand wordless and enigmatic messages. However, these signals evoke some feeling such as briskness and spookiness even though they do not know these actual meanings. If there are stimuli which allowed people to share same or similar feelings, it will be wordless word.

Relationship between specific external stimuli and emotions are recorded in a comparison table for translation from feedback to next stimuli in Step 3.

Step 2 : Representation of feedback from the brain
Next step is the conversion from brain reaction to cognitive output such as visualization and sounds. EEG headset reads four different kinds of emotions strength; excitement, engagement, frustration and meditation. To demonstrate these emotions effectively, the installation utilizes the research of synesthesia.

Synesthesia world
External stimuli such as sound, visualization and smell are applied to specific sensory receptors. However, some people’s brains apply stimuli to different modality and cognitive pathway as they received stimuli. This neurological phenomenon called synesthesia shows that sensory discordance brings people to unusual impressive experience. According to Daniel Hammet who is well known as an autistic savant and “the Brain man”, he feels color, emotion and personality when he sees words and number.

Even though synesthesia is a controversial phenomenon in terms of verification difficulties, the basic idea that one stimuli provide unusual sensory reactions. For instance, Wassily Kandinsky drawed music composition on canvases. This feedback needs to be cognitive and suggestive for other audience (visualization and sounds might be feasible). Furthermore, emotion kinds and value size are significant for next step.

Step 3. Coding brain’s feedback to stimuli
The final step is coding brain’s feedback to stimuli for next participants. It is similar to translation in different languages. In this step, installation team uses the comparison table of stimuli and emotions. According to the table, feedback messages are translated into stimuli for next audience. As the table is based on hypothesis, it needs to be modified and improved continuously during numerous experimentations. After translation, new message made by previous brain’s feedback will be transferred to newt audience.

References

  • Chatwin, B. 1987. The Songlines. London. Jonathan Cape.
  • Classen, C. 2005. McLuhan in the rainforest: the sensory world of oral cultures. In D. Howes (ed.), Empire of the Senses: The Sensual Culture Reader: 147-163. Oxford. Berg.
  • Harrison, J. 2001. Synaesthesia: The Strangest Thing. Oxford. Oxford University Press.
  • Ingold, T. 2000. The Perception of the Environment. London and New York, Routedge.
  • NASA. 2014. Golden Record [Online] Available from: voyager.jpl.nasa.gov/spacecraft/sounds.html [Accessed: 26th February 2014].

Music and Emotion – Submission 1 Essay

DMSP Submission 1 – Music and Emotion

 The purpose of this year’s Brain Drain DMSP project is to use electroencephalography (EEG) readings to monitor an individual’s emotional reaction to stimuli – stimuli in this case being sound. An EEG headset will measure brain activity, which is decoded into numerical values between 0-1 for four separate moods; engagement, meditation, excitement, frustration. These ‘mood’ readings are based on the strength of the alpha, beta, theta, and delta brainwaves. These numerical values will then be used to control visual output, hopefully providing insight towards the individual’s emotional response to sound. The sound will be ‘composed’ in real-time, with a level of user-control being provided to the audience. The conclusion of the project will be an interactive installation combining each of these elements, with the aim to create an immersive and enjoyable, yet informative, experience.

This report will discuss the main theoretical concepts underlying the project, with a focus on sound and its intended emotional effect on the user. Decision-making regarding sound will be supported by studies in the field linking music/sound and emotional/cognitive response. Installation setting and audience participation will also be discussed, considering a way to include an element of user-control that allows for freedom and enjoyment for the audience, but ensures an interesting and effective installation.

“All musical emotions occur in a complex interplay between the listener, the music, and the situation.” (I. Deliège, pg 118)

Before analysis of sound and how it may be used to evoke emotion, consider the point made in the above quote. Emotional response to music or sound cannot simply be thought of as a direct result of a specific timbre, phrase, or sequence of notes, but that of a range of factors. Therefore, to reach the goal of making the stimulus effective in this project, looking beyond the sound itself is essential. Although sound is the main stimulus for the performer, the location and audience participation may play as vital a role.

The size and shape of the performance space will determine how the sound will travel and reverberate. Size and shape, along with building materials, determine dampening/attenuation of specific frequencies within the audible range due to standing waves formed between hard, parallel surfaces, and absorption of softer surfaces. Audience participation may interfere with sound or distract the performer, preventing meditation and engagement or evoking frustration. “A significant proportion (approximately 40%) of musical emotion episodes seem to occur when the listener is alone” (I. Deliège, pg 119), perhaps suggesting that separating the audience and the performer may be the best way to obtain strong emotional response in the installation. Deliège also suggests “musical emotion episodes are most prevalent in the evening” and “more frequent during weekend days than weekdays”, highlighting time as another important factor in perception and emotional response to sound.

Undeniably, we must tread carefully when designing sound with the intention of inducing a reaction in a setting that has largely been undefined. This said, there are a number of sonic features that we can look to emphasize in order to lead the performer towards a certain mind state.

“Changes in basic acoustics attributes such as loudness, tempo, and pitch height can give rise to dramatic changes in arousal” (W.F.Thompson, pg 124)

Thompson’s observation, along with Juslin and Västfjäll’s BRECVEM model, would suggest that surprise or a sudden change in sonic attributes is a valuable tool in trying to produce abrupt mood changes in the user, allowing for a more dramatic, or less stagnant, visual output in the case of this project. Providing opportunity for fluctuations in sonic characteristics such as rhythm, timbre, dynamics, and pitch of the stimuli would be a useful tool in ensuring similar fluctuations in EEG data. The next thing to consider is how to restrict the entire soundscape, ensuring that individual characteristics are not lost in a sea of sound. This is where setting and audio interface design comes into play.

Clearly, the interface design will play a vital role in the mood management of both the performer and the audience, but the setting of the event must also be chosen and arranged with as much consideration. The way the sound objects are positioned around the room, the lighting, the distance between the audience and the objects, and the location of the performer, amongst other attributes, are of vital importance in shaping the correct mood of all involved in the exhibition. Atmosphere plays a major role, and being in an unfamiliar location surrounded by strangers is to have an effect on the audience’s likelihood to interact freely with the sound objects.

The idea of arranging the performance space as if it were a living room has been raised amongst the group. This could fit in with some of the objects already being used (piano, turntable, speakers, laptop, tables), allowing for a more inviting and relaxing atmosphere. The introduction of lamps for illumination of the individual sound objects would also tie in with this theme. The individual illumination of objects in a dark room could help in attracting members of the audience to interact with these objects; something that required some encouragement during testing in a cramped, well-lit room.

Aside from all that has been mentioned, there are a range of design issues that play a pivotal role in the success of the installation; the visualization being one of the main aesthetic concerns. The visuals must be impressive on their own, but also correlate with the audio. They must be visibly linked to the output sound, not only to connect these main aspects of the installation but to provide gratification to the audience for their interaction. Providing a visible link between their actions and the sonic and visual output is the most likely way to evoke a sense of curiosity and excitement. This is seen as the best way to make the entire installation move fluently, providing a truly immersive environment for both the performer and the audience.

References

I. Deliege & J.W. Davidson, Music and the Mind. Oxford University Press, 2011.

S. Feld & K. Basso, Senses of Place. School of American Research Press, 1996.

S. Feld & C. Keil, Music Grooves. Fenestra Books, 2005.

Y.H. Hang & H.H. Chen, Music Emotion Recognition. CRC Press, 2011.

D. Howes, Empire of the Senses: The Sensual Culture Reader. Berg Publishers, 2004.

W.F. Thompson, Music, Thought and Feeling: Understanding the Psychology of Music. OUP USA, 2008.