Blind musicians and teachers communicate through vibration using Canadian-made device

As It Happens5:32Blind musicians and teachers communicate through vibration using Canadian-made device

A PhD student in Ottawa has developed a device that allows blind and low-vision students to feel music cues, replacing the need to see them. 

Instead of having to follow a conductor's baton, or a teacher's hand gestures during a lesson, music learners can receive instruction through a vibration in their right leg sent by the Tap-Tap device. 

Leon Lu, the information technology PhD student at Carleton University behind the idea, says the wearable device is easy to use and cheap to make. 

Two people wear paired devices around their right ankles. Then, by tapping their heels together in different patterns, they can send messages to each other. 

"Imagine, kind of like a Morse code, but vibration," Lu told As It Happens host Nil Köksal. "What's really cool about that is that you can get real-time communication happening through discreet methods." 

"The student can continue playing and they don't have to stop and actually discuss [the instruction], but they can still be in that moment and in flow." 

WATCH | Musicians demo Tap-Tap device

Blind violin student communicates wearing vibration device on leg

4 months ago

Duration 0:34

A violin teacher taps her heels together to send a real-time message to her student about rhythm and tempo. The device they’re using, called the tap-tap, was created by Carleton PhD student Leon Lu in an effort to transform the learning experience for blind and low-vision musicians. (Submitted by Leon Lu)

The Tap-Tap device doesn't come with a set communication code or language. That language is set by the participants, which Lu says is both a limitation and advantage of the device. 

Piano teacher Nora Bartosik echoes that.

Bartosik is a faculty member at Manhattan's Filomen M. D'Agostino Greenberg Music School, a school for blind and low-vision students, which teamed up with Lu to test the prototype and provide his team with weekly feedback about the device. 

She used the Tap-Tap device with one of her piano students for about two months earlier this year and says there was a learning curve. 

"It kind of feels like a phone buzz," said Bartosik. "Actually, a little bit stronger." 

"[My student and I] decided to focus on the things that I've been saying a lot in lessons.... We decided we would use the buzz to indicate that I wanted him to play quieter in certain sections." 

As they got used to the device, they added more signals. 

"Two buzzes might mean like get louder, you know, do a crescendo," she explained. "One long buzz would mean this or one short buzz would mean that. We kind of expanded the vocabulary as we went along." 

Chase Crispin, a blind musician and teacher in Lincoln, Neb., who consulted on the Tap-Tap project, said in a written statement from Carleton University, that "many people don't realize how much a musician is managing at once: posture, notes, rhythms, dynamics — the list goes on. For blind music learners, who memorize most of this, it adds even more layers." 

Crispin said getting involved with the study was a way to "blend my own interest in technology with the needs I had as a blind musician."

Lu, who sings, plays guitar and is learning violin, said his passion for music and desire to be creative was part of the reason he took up this project. 

He's surprised that a device similar to the Tap-Tap isn't already integrated into commercially-available devices, such as a smartwatch, and hopes that might be a possibility in the future.