Spectromorphology and Audio-Visual Performance

Real-time audiovisual performance is governed by a multitude of variables that include, but are not limited to, performance model, technology integration and aesthetic goals.  During the course of our Audio Visual Ensemble’s digital media studio project, time and time again we were asked, “What makes a convincing audiovisual performance?”  Although this is a multifaceted topic with no single definitive answer, I will offer suggestions as to how spectromorphology can be used to analyze strategies for better developing convincing real-time performance.

To start, it is important to realize that in performance situations involving interaction between performers, the audience and their medium, what viewers perceive is not necessarily the same experience as for those performing. What may deem entertaining to the performers may be misunderstood or uninteresting to the audience, especially if the performance lacks a clearly defined theme and significant dynamic change over time.  Since my background is in sound composition and performance, I have been able to draw upon some parallels of similar concerns in electroacoustic music that assist in directing this discussion.

Like language, audiovisual content is constructed from extrinsic-intrinsic building blocks and the ability to interpret their message relies on the ability to communicate meaning effectively. Denis Smalley introduced the concept of spectromorphology as a tool to assess criteria for selecting sound material and organizing structural relationships that are linked to recognizable shared experiences outside of music (Smalley 1997).  Smalley explains, “How composers conceive musical content and form- their aims, models, systems, techniques, and structural plans- is not the same as what listeners perceive in that same music.”  This begs the question, what are universal shared experiences and how might these ideas be conveyed in the realm of audio-vision.

Smalley believed that for performances which rely heavily on technology, where gesture and the corresponding results are not immediately apparent, the ability to adequately convey meaning is impaired by the inability to directly link action to a corresponding result (Smalley 1997).   He suggests that we should try to ignore the technology used in making music, and in this case, the performance, by recognizing that gesture and relationships between source and cause are a more substantial factor for communicating ideas effectively (Smalley 1997).

In a post-digital era, this is no easy feat as technology manifests on almost every level. Perhaps the paradigm has shifted since Smalley developed his initial ideas and now it is more salient to balance technology in ways that convey universal shared experience.  I propose that technology alone is not the underlying concern but that relying on technology without having developed a well-defined theme or aesthetic goal tied to significant meaning or gesture is of greater importance.

Spectromorphology is concerned with motion and growth process and points to trans-contextual interactivity of intrinsic and extrinsic relationships for answers.  These intrinsic-extrinsic relationships need not be limited to sonic or visual events themselves but can be extended to the gesture that defines them (Smalley 1997).  Smalley refers to this concept as source bonding, which relates sound structures to each other on the basis that they appear to have symbolic or literal shared origins.  Applied at both higher and lower levels of structure, physical or metaphoric gestures that clearly exemplify corresponding sonic and visual counterparts can be used to more clearly convey meaning, purpose and intent (Smalley 1997).  Translating these ideas to the audio-visual realm means linking sound and vision in ways that dynamically delineate cause and effect.  By extrapolating intrinsic perspective and translating meaning into extrinsic constructs, both the performers and audience gain deeper insight into function and significance of not only the main theme but also the individual contributing factors that make up the performance.

Whilst meaning plays an important role in disclosing intention, it cannot be convincingly conveyed without placing it into a larger structural context.  Smalley explains, “Motion and growth have directional tendencies which lead us to expect possible outcomes.”  He in turn breaks these into seven characteristic motions that relate to spectral relationships of sound production (Smalley 1997).

Motion Chart

Although it is beyond the scope of this brief essay to elaborate on each of these, it is important to note that these structural relationships can be applied to help establish dynamic trajectory in ways that strengthen structure to better convey meaning over time.  If applied to audio-visual performance, we can devise precise ways of mapping movement to states of tension and release based off of relationships dictated by transitioning events in conjunction to their starting points.  Extrapolating on this idea, if we identify events according to gesture, movement and meaning, we can fit them into a larger dynamic structure and manipulate their placement to either fulfill or break expected outcomes.  This approach in turn can be used to create a more systematic approach and aide in building more elaborate and well though out performance structures.

Stepping back from our performance and having time to contemplate its strengths and limitations, it has been beneficial to consider concepts pertaining to spectromorphology and how they could be applied to our work.  To a degree, because combined audio-visual performance was new to us, the group was short sided in its approach by placing too great of an emphasis on developing the technology used therein.  Although we settled on a theme, overall specific details failed at times to directly correlate to any shared experience outside of the sounds and visuals themselves.  The level to which our performance could grow, change and evolve may be improved with more attention to how we guide transitions through directed movement, energy and gesture.  While spectromorphology does not provide a universal set of answers, it does afford some valuable tools for diagnosing structure and strategy.


Smalley, Denis. 1997. “Spectromorphology: Explaining Sound-Shapes.” Organised Sound 2 (2): 107–26.

Leave a Reply