Back to stories Back to stories

Eye Gaze Technology and Music

What is Eye Gaze and how does it work?

Eye Gaze uses the technology of eye tracking, which monitors what people do with their eyes. They measure how our eyes move, the direction they move in and the gaze points to determine where the user is looking. It’s most commonly used to analyse visual paths when the user looks at a computer screen. For example, we could see how you’re reading this blog right now. Did you skip the paragraph above? Are you just looking at the images? How long did you take to read the next paragraph? This great video gives you an idea of how tiny the movements that can be monitored are (you can choose to ignore the ever so subtle marketing effort right at the end…)

Thankfully eye tracking technology has now become non intrusive so there’s no need to jam ‘Terminator’ style gadgetry into your eye socket (sorry for anyone with particularly bad eye-related squeamishness) and works using special lenses and video cameras. They’re mounted under the computer screen and illuminate the eye using a low power infrared LED that reflects light off the cornea. It also reflects off the retina (the back of they eye) causing the pupil to appear white for the camera, allowing it to track the gaze much more accurately and capture 60 images within a single second. For people that are physically disabled or have restricted mobility, it’s become an instrumental tool for communication. It’s particularly used to help those with Cerebral Palsy as well as Muscular Dystrophy, Spinal Muscular Atrophy, Multiple Sclerosis and spinal cord injuries. The user can simply select spoken phrases they want to say by ‘dwelling’ on them on screen for a duration of time and a speech generating device will speak it. It can also be used to navigate and select what’s on screen much like a mouse for a regular computer. Pretty clever stuff.

So what does this have to do with music?

Over the last few years, there have been multiple software programs developed to utilize EyeGaze for creating music as well as speech, so now users can express themselves through music too!

Much like selecting phrases for speech devices, users can select instruments and the notes they play to then build their own song from scratch. The technology is far from basic and offers a huge amount of customization that really lets the creative juices flow. Notes can be played at different lengths and be built up to create full chords, pitches can be shifted, loops can be created. Once the user’s put together a bass line, beat and synth rhythm for example, they can then be put together much like how you would record music in Pro Tools or Garageband software. Check out this piece over at Drake Music where you can see the program put through its paces. And that’s just one piece of software! There are programs for individual instruments like drum kits and harps for musicians to get more specialist with the instrument of their choice.

It’s not limited to just recording music either. Organisations like the Open Up Youth Orchestra are allowing disabled musicians to perform live using various other technologies and instruments. You can check out the amazing work they do on their website.

It’s amazing to see that technology is being used not just to improve existing music products, but to include and encourage EVERYONE to get involved no matter their physical or mental limitations. We’re strong believers in the power of music to medicate the soul, so looking forward to seeing what the future has to hold for this technology. We might not have to wait too long before it's a universally used piece of technology for both able and disabled users: