A semiautonomous robotic prosthesis has turned an amputee drummer into a three-handed “cyborg”.
The robotic drumming prosthesis has motors that power two drumsticks, the first of which is controlled both physically by the musicians' arms and electronically using electromyographic (EMG) muscle sensors, while the other stick "listens" to the music being played and improvises.
The device is the brainchild of Georgia Tech Professor Gil Weinberg, who has already built a band of robotic musicians, and has been tested on Jason Barnes, a drummer who was electrocuted two years ago and lost his right arm below the elbow.
"The second drumstick has a mind of its own," said Weinberg, founding director of the Georgia Tech Center for Music Technology. "The drummer essentially becomes a cyborg. It's interesting to see him playing and improvising with part of his arm that he doesn't totally control."
Barnes, a student at the Atlanta Institute of Music and Media, built his own prosthetic device shortly after his accident, but he found it wasn't very flexible. He could bang the drums by moving his elbow up and down, but couldn't control the speed or bounce of the stick without a wrist or fingers.
That's when Weinberg stepped in to create a single-stick device with sensors that respond to Barnes' bicep muscles.
"Now I can flex and send signals to a computer that tightens or loosens the stick and controls the rebound," said Barnes.
But Weinberg, who has already built a robotic percussionist and marimba player that use computer algorithms to improvise with human musicians, took the prosthesis a step further by adding a second stick and giving the device a "musical brain".
"Jason can pull the robotic stick away from the drum when he wants to be fully in control," says Weinberg. "Or he can allow it to play on its own and be surprised and inspired by his own arm responding to his drumming.
“Music is very time sensitive. You can hear the difference between two strokes, even if they are a few milliseconds apart," added Weinberg. "If we are able to use machine learning from Jason's muscles (and in future steps, from his brain activity) to determine when he intends to drum and have the stick hit at that moment, both arms can be synchronised."
Weinberg says such robotic synchronization technology could potentially be used in the future by able-bodied humans to control an embedded, mechanical third arm during time-sensitive operations. For example, Weinberg's anticipation algorithms could be used to help astronauts or surgeons perform complex, physical tasks in synchronisation with robotic devices.
For Barnes, it's all about the music. Because an embedded chip can control the speed of the drumsticks, the prosthesis can be programmed to play two sticks at a different rhythm. It can also move the sticks faster than humanly possible.
"I'll bet a lot of metal drummers might be jealous of what I can do now," he said. "Speed is good. Faster is always better."
Barnes will play with the device for the first time publicly on 22 March at the Robotic Musicianship Demonstration and Concert at Kennesaw State University's Bailey Performance Center. The free event, which will also feature Weinberg's Shimi robot, is part of the Atlanta Science Festival.