Reactive Hybrid-Ecological Audio Enhanced Interface

Caolan McNamara B.Eng, Interaction Design Centre, University Of Limerick

Email: Caolan.McNamara@ul.ie
Phone: +353-61-202699

The Sound of Action Project, is an attempt to enhance the standard graphical desktop interface using audio. Sound is ideally suited to notifying the user of the results of his actions, to express errors and warnings and to monitor background activities. Audio can be used to reinforce the feedback currently given in graphical form, to help minimize confusion over the meanings of the visual cues.

There are two basic branches of audio display, earcons and auditory icons. Earcons are structured and abstract sounds whose meaning must be learnt, earcons consist of an easily generated sequence of tones and chimes and has an internal grammar and structure of its own. Auditory Icons are ecologically tied to the event, the sound they create maps naturally to the event that created them. i.e. an auditory icon representation of a button would make a click sound that matches what the real-world counterpoint of that button might make, whereas an earcon would make a musical sound of some kind. The trade off is that earcons are more difficult to recognize and learn, while auditory icons are difficult to parameterize and generate, but are easily understood and recognized by users.

Parameterization is an important part of an audio interface, the audio feedback of the environment should match the action and activities taking place in the environment to create any added value. It is quite easy to parameterize an earcon, as the sound of an earcon can be derived from a set grammar. Auditory icons are messier to parameterize, it is difficult to artificially generate an appropriate natural-like sound from a given scenario.

What SOAP is attempting to do it find a middle way, a hybrid, between these two approaches. To create a system that uses a simplified framework of sounds that allow us to generate auditory icons that while not perfectly natural sounding, like true auditory icons, have the key invariants that users use to identify the nature of real world sounds. As an example, the key invariant in judging the speed and distance of a car from its sound is the rate of increase of its pitch, the other elements of the sound are non-essential in determining this information. Just so, we hope to create a range of skeleton sounds that we can use in the desktop to harness the natural strengths of the users listening skills to enhance the existing graphical environments.