allenaeroplane: Virtuoso cartoon pianist plays just like the real thing

Monday, September 24, 2012

Virtuoso cartoon pianist plays just like the real thing

Hal Hodson, technology reporter

Watch a professional pianist play and you'll be struck at how effortlessly their hands skip across the keys.

Making an animated character do the same is much harder, so researchers from the University of California, Davis, have built an algorithm designed to make computer-generated hands tickle the ivories far more realistically.

Their software auomatically generates a 3D animation of piano-playing hands based solely on inputted music. The software analyses a music file, then builds a pair of animated hands playing the song. Developed with colleagues by Davis computer scientist Yuanfeng Zhu, the system is based on the idea that virtuoso pianists will always move their hands in a way that relaxes their body and saves energy.

To achieve this using software, Zhu calculates the most efficient path for the 3D fingers to take over the keys. Finger positions are calculated for every chord in a piece of music, as is the motion of each hand throughout the piece. The algorithm also takes the position of every finger into account at all times, even when that finger is not involved in the next chord. This ensures the animated hands play the music in as relaxed a manner as possible.

Although the 3D hands look rather uncanny, their motion matches a human player rather well. The researchers suggest that their technique is realistic enough to be used to self-teach the piano, by using motion capture software to compare your performance with the algorithm. The paper notes that it would be possible to apply the animation to other instruments, such as violins and flutes.

"Although our system can generate plausible and reasonably realistic piano playing for standard music, it is not capable of generating emotional piano playing that re?ects a personal understanding of the music and player's performance background," the paper concludes. "This is the main future work we will be pursuing."