By Nicola Oddy
Context
The theme that my team is addressing is Social Inclusion – physical. Music making is a social activity and requires physical input. As a music therapist, I have never worked with technology but understand the benefits of these tools. This project has inspired me to begin creating a comprehensive database of existing products, and to review them from the perspective of someone who works with people who have disabilities and who help those people with social inclusion and physical goals.
In this blog post I begin my study by exploring eight tools that could be used to create social opportunities and require a range of physical capabilities from fine motor control to full range of motion. In music therapy, clinical improvisation is used as an important technique to help clients with their physical, emotional, psychotherapeutic, and social goals. Here I am focusing on social and physical goals, but all of these tools could help clients to improvise in ways that could benefit them from a larger perspective.
I was fortunate to have the opportunity to interview Henry Lowengard (designer of the iOS version of AUMi) and also Susan LeMessurier Quinn, Music Therapist who uses this tool in her work.
Adaptive Use Musical Instrument (AUMI)
Adaptive Use Musical Instrument is one example of an accessible music technology application that can be used to help people with disabilities experience social inclusion by participating with the full range of their physical ability in music making. Henry Lowengard adapted this program for iOS to make it more accessible to a broader range of participants. From the physical perspective, AUMI is based entirely on movement and accommodates a wide range of ranges of motion and ability to pinpoint a smaller section or not
The development of AUMI grew out of a need when occupational therapist Leaf Miller was working with a drummer who had severe mobility issues, and wanted to help students with disabilities participate in music making. In 2007, she asked Pauline Oliveros for help and a team was created to tackle this. They worked with the actual students in question to create an instrument that would adapt to anyone. It was originally created by Zane Van Duzen and Zevin Polzin who were students at Rensselaer Polytechnic Institute (RPI) and further adapted for iOS by Henry Lowengard.
AUMI is distinct from other accessible instrument technologies in that it does not require switches or an infrared sensor mechanism to activate sounds. Lowengard uses as little redundancy and as much simplicity as possible in his design. It works through a video tracker.
There are two users with AUMI–and I suspect with most accessible music technologies – that of the end user and that of the facilitator. The end user that AUMI focuses on often has cognitive or physical challenges. It takes a certain level of cognitive and physical ability to set up AUMI for improvisation or performance. Accessibility is focused on the actual playing of the instrument after a facilitator has set it up.
Sounds, modes, instruments, volume and visual elements are set up by the user or an assistant. Here, the word ‘mode’ is referred to in two ways: That of the type of scale used (major, minor, dorian, etc), and as the way in which different kinds of musical improvisation, and ways to make sounds are accommodated. For example, AUMI uses the user’s motion, size of active part of the screen, and size of the responsive part of the body to create sound. While some instruments play notes, others are percussive or play sound effects, and there is a way to play sounds using the relative motion of the user. The user can then move a body part in front of the instrument to make sounds just by moving. The area on the screen that is active can be made larger or smaller in order to provide ease or challenge for the user, depending on what the goals are for that individual. The area that creates the sound (the dot on the body part) can also be increased or decreased in size to help the player participate successfully.
It is possible that there was nothing comparable on the market in 2007. Since then, a strong representation of music technologies has emerged around the world, that would be suitable for use with people who have a full range of abilities. Here are some examples.
Using AUMI as a Tool in Music Therapy
I decided to deepen my study by speaking directly to a music therapist who uses AUMI.
Susan LeMessurier Quinn is the music therapist at the Janeway Children’s Health and Rehabilitation Centre in St. John’s Newfoundland. She was involved in a research project in which she participated by using AUMI as a tool for therapeutic intervention (Finch, LeMessurier Quinn, Waterman, 2016) On December 4, 2017 I had the opportunity to speak with her about her work using this technology in a therapy setting. The research was published in 2016, but the sessions were conducted in 2012. At the time of the research, the program had not yet been developed for iOS and was still in development for PC.
A music therapist works with each individual, with specific therapeutic goals. Susan feels that this tool is a good one to have in her kit. With each person a music therapist works with, the instrument that best suites the person, the goals and the moment is chosen. She acknowledges that every child may not benefit from it (depending on goal and setting) but that it is often useful. For each child that it is used with there is an assessment process to determine its efficacy. Then, if it is useful for that child she will determine the mode, sensitivity, size of the reactive part of the screen, size of the dot, the square and other adaptive elements that AUMI offers.
During the research project, she noted that it was a good tool for physical, social and cognitive goals. When she was helping a client to strengthen his neck, she found he would hold his head up in order to hear the changes in sound. When he moved his head and made sound on AUMI, Susan improvised with her flute to help him understand that he was a part of the music making. She imitated his sounds and he responded by keeping his head up. When using it to encourage vertical movement, a client would sit up tall and posture improved. It also enabled a child to make a connection and to know when the sound was being made by him. It helped a teenager who could achieve social independence by using the tool. Cognitively, it helped children with cause and effect.
Susan continues to use this tool, encouraging a client with spastic Cerebral Palsy to engage in independent movement. She has acted as a consultant to teachers who use it in the classroom for those who cannot participate in music making otherwise.
She finds that with her clientele who often have cognitive delays, a volunteer is needed in the group setting to help the child make the connection between their movement and the sound and they usually need time to understand that this, before beginning an improvisation.
Here is the research article. To view a video, scroll down around half way.
Using AUMI as a tool in Community Music
AUMI is used along with other music technologies to engage members of the community to participate in music making. The mission of H’art Centre in Kingston and Ottawa is “to offer high quality opportunities for people with disabilities and those facing barriers to create, study, and produce works in the arts.” (H’art School of Smiles, 2017). Through organizations like H’Art and the Centre for Human Development in the Arts at Concordia University, (there is a helpful video if you click on the link) members of the community are given the opportunity to make music and perform. Engagement with tools like AUMI help those who have more difficulty participating, and creates an environment of inclusivity. As stated by Jesse Stewart “…music is a fundamental human right and everyone deserves to have opportunities to make music, regardless of musical training, social-economic circumstance and/or level of physical or intellectual ability” (Stewart, 2017)
I have explored seven other accessible music technologies which are effective as tools for people to use in therapy, performance, or just for fun with friends. Many of these tools are free or cost very little. This is the tip of the iceberg – there are dozens of tools like these available.
Virtual Music Instrument
Andrea Lamont, music therapist, along with 3 biomedical engineers, developed the Virtual Music Instrument at Holland Bloorview Kids Rehabilitation Hospital in Toronto. To deepen my understanding of this tool I asked Andrea to speak to me about her work with it. We met by Skype on November 23, 2017. She spoke with me about the development of this instrument which took place between 2002 and continues to this day. It is available for PC at this time and requires an external camera. The next step is to create an application. It is free on Flintbox.
Usability studies were done for several years to tweak the technology and many different versions have led to what exists now. Requirements for Andrea are that 1) it needs to be very simple so that in the context of a therapy session it doesn’t stop the therapeutic flow 2) needs to be adaptable to the needs of each child, especially in terms of sensitivity.
The way it works, is that shapes are selected and a decision is made about where on the screen-space they should go. A portion of the screen is assigned to a certain sound when the shape is placed there. Shapes that activate the movement have instrument images in them. The child can see the circle and the instrument and when they move parts of their body, they activate the sound. This gives them a sense of ’cause and effect’. When the pixels are disrupted in that space, and movement is detected, it makes a sound. The session facilitator can change the frequency and the instrument from a standard bank of midi sounds or load a custom sound. ‘Dwell time’ can be adjusted, so that a sound can be held for longer or shorter periods.
Sensitivity is an important factor. For example, if someone does not have much motor control, sensitivity can be lowered so that frustration is not caused by the sound being activated when it is not intended. Clear movement then, activates the sound. For a child with a small head turn as their primary movement it can be made very sensitive. One person that Andrea told me about, could only make an eye blink, and the sensitivity was set very high so that his blink activated the sound, which caused a very positive response in the individual.
It can be projected on the wall of the gym so that many children can run across the image to make a group improvisation. It can be projected onto the floor so that children can jump on virtual objects to make sounds, or on the ceiling so that those who are more stable when lying down can participate.
Andrea and her team received the Governor General’s Meritorius Service Cross on Dec. 12 2017 to honour them for this creation The decoration recognizes Canadians for exceptional deeds that bring honour to the country. See an article about this event;
Skoog Music
Skoog Music is out of Edinburgh is a tactile device that works with an application that changes the sounds and allows for inclusion. A cube with speakers on all but the side resting on the table allows musicians can touch any side to create sounds. A child would need to have the ability to press the flexible buttons, but what they press would not matter. The music therapist or musician could work with the individual to create a successful musical outcome. www.skoogmusic.com
Thumb Jam
Thumb Jam is an iPad app which allows for the playing of many instrument options. It presents with vertical lines from left to right. It’s a very simple concept that requires touch right on the iPad. This would be good for hands on ‘action/response’. A touch of the iPad creates the sound of the instrument and the response can be adjusted. Here’s a demo to watch. https://www.youtube.com/watch?v=kvlv7OF5pYk
Bebot
Bebot by Normalware, is a fun application that incorporates humour as the player makes the little animated robot vocalize in all sorts of ways. The principle of left to right vertical lines is similar to Thumb Jam, but the little robot does all of the sound making. It requires a direct touch on the screen. Take a look at a demo a the following URL: https://sites.google.com/site/musicandspecialeducation/home/resources/related-website/electronic-music/bebot—robot-synth
Launchpad
Launchpad is an instrument that allows the player to touch buttons on a grid and to create music. The demo at the URL https://global.novationmusic.com/launch/launchpad shows a sophisticated use of this instrument but I can see how a person with disabilities could enjoy a tactile experience and be able to understand the relationship between the action that they make and the sound that emerges. A real instrument would do the same thing however.
Ikaossilator
This application allows for the creation of loops. It’s a sequencer that would allow a player to join in with bass line or rhythm. A user would need help to set this up if they did not have technology know-how but once that’s done I can see how it would be fun to use. The screen is perhaps too busy for someone who has cognitive or physical challenges. http://www.korg.com/us/products/software/ikaossilator/ Scroll down to the bottom of the page to see videos.
Airvox
Airvox is very similar to AUMI. It responds to hand movements and the player does not need to come in contact with the screen. This product seems a bit busy in terms of screen information. The dot that follows the hand has the note that it is playing embedded in the image and the controls are always visible. This is a playful device. https://www.youtube.com/watch?v=U625-iic1QE
Adapting Existing Instruments
Music technology does not need to be digital. Musical instruments can be adapted for use by people who may not be able to play traditionally. This little girl was born with Arthrogryposis multiplex congenital, meaning that some of her joints can’t move and prevent her from participating in all of the activities that she would like to be a part of. The video comes from www.flowmarching.com. She wanted to join the band, and her family had a Euphonium adapted so that she could play it with very little pressure. The instrument is held in place with a stand and she plays with light touches. https://www.flomarching.com/video/6049998-where-music-and-engineering-meet
Coda
As a music therapist, my aim is to help people make music in ways that will enable them to participate, experience success, and rise above the difficulties that they experience in their lives be they physical, social, emotional, psychotherapeutic, cognitive, or communicative.
Making music is the means to an end and it’s used in whatever way that’s needed in order to help that individual. Singing, playing instruments, writing songs, listening to music together, moving to music, talking about music, and analyzing lyrics are a few examples of ways that I can help.
Music therapists, inherently work with accessibility through adaptation of traditional instruments, and the creation of ways to make it possible for a person to engage with us. It might mean devising a stand for a chime, or padding a drumstick handle. It may mean a simple repositioning of an instrument, or designing it to respond to a need such as in Chloe’s case.
Music technology is an added tool that a music therapist can use to help with Social inclusion from a physical lens, among other goal areas that a client may need help with. Nothing could detect the movement of an eye-lid until Andrea Lamont learned that the Virtual Music Instrument can do just that. Spontaneous response to movement that AUMI provides enables children to participate through simple movement and it plays a big part in music therapy, community music, and everyday music-making of many people.
References
Scholarly articles
Finch, M., LeMessurier Quinn, S.,& Waterman, E. (2016) Improvisation, Adaptability and Collaboration: Using AUMI in Community Music Therapy. Voices: A World Forum for Music Therapy, 16(3). Retrieved from https://voices.no/index.php/voices/article/view/834
Hunt, A. & Kirk, R. (1997). Technology and Music: Incompatible Subjects? British Journal ofMusic Education, 14(2), 151-161.
Knox, R., Lamont, A., Chau, T., Hamdani, Y., Schwellnus, H., Eaton, C., Tam, C., and Johnson, P. (2005). Movement-to-music:Designing and implementing a virtual music instrument for young people with physical disabilities. International Journal of Community Music. R(1). Retrieved from http://www.intellectbooks.co.uk/MediaManager/Archive/IJCM/Volume%20B/06%20Knox%20et%20al.pdf
Lem, A., & Paine, G. (2011). Dynamic sonification as a free music improvisation tool for physically disable adults. Music and Medicine. 3(3), 182-188 doi: 10.1177/1943862111401032
Magee, W.L., &Burland, K. (2008). An exploratory study of the use of electronic music technologies in clinical Music therapy. Nordic Journal of Music Therapy, 17(2), 124-141.doi: 10.1080/08098130809478204
Nagler, J. (1998). Digital Music Technology in Music Therapy Practice. In C. Tomaino (Ed.), Clinical Applications of Music in Neurologic Rehabilitation, (pp. 41-49). Saint Louis: MMB
Oliveros, P., Miller,L, Heyen, J., Siddal, G., Hazard, S. (2011). A musical improvisation interface for people with severe physical disabilities. Music and Medicine 3(3), 172-181. doi: 10.1177/1943862111411924
Tucker, S., Oliveros, P., Rolnick, N., Sun Kim, C. Tomaz, C. Whalen, D., Miller, L., Heyen, J. (2016) Stretched boundaries: Improvising across abilities. In G. Siddall & E. Waterman (Eds.), Negotiated moments: Improvisation, sound and subjectivity (pp 181-200). Durham, NC: Duke University Press doi: 10.1215/9780822374497-011
News articles
Bloomfield (2016). Embracing Music Technology: The DM lab challenge British Council News. Retrieved from http://music.britishcouncil.org/news-and-features/2016-10-20/embracing-music-technology-the-dm-lab-challenge
Thomas, P. (2012). Music Technology & Special Needs: Part 1. Sound on Sound News Forum. Retrieved from https://www.soundonsound.com/techniques/music-technology-special-needs-part-1
Nagle, P. (June 2010). Korg Kaolssilator Pro. Sound on Sound News Forum. Retrieved from https://www.soundonsound.com/reviews/korg-kaossilator-pro
Websites
Airvox. (2011). Retrieved from https://www.youtube.com/watch?v=U625-iic1QE
AUMI: History. (2016). Retrieved from http://aumiapp.com/history.php
Bloorview Research Institute. (2017). Virtual Music Instrument. Retrieved from https://research.hollandbloorview.ca/innovations/virtualmusicinstrumentVirtual Music Instrument. Retrieved from https://www.youtube.com/watch?v=pS2Higl1xJs
Centre for the Arts in Human Development. (2016). www.concordia.ca/research/cahd.html
Florida State University. Music and Special Education-Bebot. Retrieved from https://sites.google.com/site/musicandspecialeducation/home/resources/related-website/electronic-music/bebot—robot-synth
H’art School of Smiles. (2017). Retrieved from https://www.hartschool.ca
Stewart, J. (2017). Retrieved from www.jessestewart.ca
Loopseque. (2011). Retrieved from https://www.youtube.com/watch?v=bNv2pCdNqJI
Korg: ikaossilator. (2016). Retrieved from http://www.korg.com/us/products/software/ikaossilator/
Skoog Music. (2017). Retrieved from http://skoogmusic.com
Novation, Launchpad. (2017). Retrieved from https://global.novationmusic.com/launch/launchpad#
Thumb Jam. (2017). Retrieved from https://www.youtube.com/watch?v=kvlv7OF5pYk
Research Groups
Adaptive Music Technology Research Group (2017). University of Huddersfield: Huddersfield, UK Retrieved from https://research.hud.ac.uk/institutes-centres/amtrg/
Adaptive use Musical Instrument Consortium. Rensselaer Polytechnic Institute, Center for Communication, Cognition and Culture. Retrieved from http://ccc-rpi.org/research/aumi/
ACCESSIBILITY AND INCLUSIVE DESIGN CASE STUDIES 