Between Barriers and Prospects: Merging Art Performance and Engineering in Mobile Music Education and Research

Coordinator: Georg Essl, University of Michigan

There is little doubt that mobile smart devices are a socio-cultural game changer. The reach of sophisticated, networked, interactive computational technology will soon be universal. This means that technology with tremendous capabilities for artistic expression open up a space of exploration for new forms of culture and creativity.

However, this potential is as yet to be realized. Currently we face a barrier of access for most users of mobile technology due to the complexity and sophistication of the devices and the depth of domain knowledge required to build creative applications on them. Typical users of smart devices are not trained engineers, nor necessarily trained performance artists. Hence we see the continued effort of lowering the barrier of entry as a critical aspect of the advent of mobile smart device adoptions for creative expression.

Mobile smart devices are also shifting various technological paradigms. Creative content creation on laptop and desktop computers assumed a given interaction model centered around keyboard, large monitor, and mouse. A multi-touch centric device with a small display and additional rich input sensors such as cameras and motion sensors replaces this. Hence existing models for supporting computational creativity have to be rethought and fitting models of Human Computer Interaction need to be developed.

To this end we develop an environment called urMus, which seeks to provide a mobile-centric design of open, and accessible creativity support. In the paper we will discuss the state of the project, some past experience in using it in workshops and performance. Finally we discuss a range of open challenges in research, dissemination, and adoption. For example, industry is cautious of giving open access to their devices for reasons of allowing monetization and to combat security concerns. This interferes with open creative expression and its dissemination. Building awareness and advocacy are important to mitigate some of the impact of these problems.

Further there is a need to rethink curricula. We designed a senior level undergraduate course titled “Mobile Phones as Musical Instruments.” It is cross-listed between the College of Engineering and the School of Music, Theater and Dance at the University of Michigan. The placement of such interdisciplinary course has numerous challenges but also clear benefits. The course is designed to blend students from diverse preparatory backgrounds. All students engage in the full range of activities in the course without distinguishing if they pursue education in engineering or the arts. We discuss describe advantages and pitfalls of this course design. UrMus is the central programming platform in our course. Its design allows rapid access, early rewards, and a sense of mastery. As students become more proficient the design allows deep engagement and open expression. An exit surveys show that students largely see the approach as successful, independent of prior background.

Mobile smart devices have already had a drastic impact on how we use computation and expanded who is able to participate. The prospect of enabling broad participation on technological creativity is tremendous with potential impact on who can enter STEM and creative fields.