Using Embodied Cognition to Improve Music Interaction Design
As recently featured in the Computer Music Journal Special Issue on HCI.
In domains such as music, technical understanding of the parameters, processes and interactions involved in defining and analysing the structure of artifacts is often restricted to domain experts. Consequently, music software is often difficult to use for those without such specialised knowledge. The present work explores how the latter problem might be addressed by drawing explicitly on domain–specific conceptual metaphors in the design of user interfaces.
Conceptual metaphors are used to map image schemas, structures which are rooted in embodied experience, onto, potentially unrelated, abstract domains. These conceptual metaphors are commonly, though not exclusively, identified through linguistic expressions in discourse and texts. We hypothesise that, like all kinds of human understanding, both expert and novice understanding of musical concepts is grounded in conceptual metaphors based on image schemas. We further hypothesise that if we can identify the conceptual metaphors used by music experts to structure their understanding of domain concepts, then we may be able to systematically improve specialised musical user interfaces to better reflect these conceptual metaphors. In this way, it may be possible for users of such interfaces to exploit the pre-existing embodied knowledge shared by all users, and to lessen the requirement for specialist domain knowledge, formal reasoning, and memorisation of technical terms. In addition, we hope to find ways to systematically improve music interaction designs in general, lessening the requirement for this specialist domain knowledge. Recently, the conceptual metaphor approach has been applied to areas including the analysis of music theory, and the improvement of user interface design, but to the best of our knowledge, the present work is the first attempt to combine these distinct bodies of research.
- Wilkie, K., Holland,S. and Mulholland, P. (2010) What Can the Language of Musicians Tell Us about Music Interaction Design? Computer Music Journal, 34(4). CMJ Wilkie Language Music Interaction Design_v2.pdf
- Wilkie, K., Holland, S. and Mulholland, P. (2009) Evaluating Musical Software Using Conceptual Metaphors. In Proceedings of BCS Human Computer Interaction, Ed. Blackwell, A. 2009 pp 232-237. EWIC, ISSN 1477-9358. BCS HCI 2009 Wilkie Evaluating musical software.pdf
(Katie Wilkie, Simon Holland, Paul Mulholland)