Exclusive Interview with Erwin Schoonderwaldt (Part 2)

Motion capture technology has helped athletes optimize their performance, improved medical treatment for physical therapy patients and made our movies look more human. But what has it done for music? Researchers are now working to bring the advances that motion capture has brought to other fields to music.

In this second conversation with Erwin Schoonderwalt, we sat down in his office to discuss his research in greater depth. He offers fascinating insights on relationship between art, performance and technology, and how motion capture technology can bridge the apparent gaps between these areas. His research offers not only practical advice to string players, but strives to help us better understand what makes a great performance truly spectacular.

—Hans Jørgen Jensen

STRING VISIONS

I am always fascinated by how sophisticated the human mind and body are. In my opinion, no machine or computer has ever come close to measuring anything with the same sensitivity that our ears and mind can. Technology helps us better understand music and help us  more quickly improve our playing, but it cannot to explain those special moments of beauty and wonder that numbers can never quantify. Overanalyzing simple things can make them too complex.

ERWIN SCHOONDERWALDT

I really like this paradox you point out.  There seems to be this inherent conflict between man and machine, because machines have no feeling for artistry. Even with technology as advanced as we have today, there are still a lot of things in art, performing and everything else that remain unexplained.

But it is also very interesting how there are certain problems that you can approach using machines, and how you can help understand these issues in a completely different way using a new perspective. A scientific approach to music performance gives you a different angle on a complex task like playing a musical instrument that players can really benefit from.

STRING VISIONS

What kind of technology do you use in your research?

ERWIN SCHOONDERWALDT

One of the main tools I use is a motion capture system, which can very precisely measure and quantify the movements we make when we play a musical instrument. I mostly work with string instruments, particularly on bowing, which is a very overt movement.  This technology is very well suited for measuring these gestures. Motion capture systems give us  a spatial resolution well within a millimeter, so we can measure points with a very high precision and at a very high speed.  With normal video, we can record 25 frames per second, but with motion capture, we can record 250 frames per second or more. The result is a very detailed illustration of the movements we make, and they reveal a lot of interesting information.

STRING VISIONS

Like what?

ERWIN SCHOONDERWALDT

At first, when you look at the data or visualize the movement, you have may have one hypothesis about what you are seeing, but when you run the data and look at the statistics, the truth about how things are moving can often come as a surprise. But then, when you start thinking about it again, you realize that there is a very natural explanation behind the data.

STRING VISIONS

How are you using motion capture in your research now?

ERWIN SCHOONDERWALDT

Right now, my focus has been going more into the body of the player. My PhD work focused on the acoustics of the instrument and the parameters of sound production. My current work deals with how the player actually controls these parameters.

STRING VISIONS

Just like they do in sports with improving an athlete’s physical movements to make them more efficient.

ERWIN SCHOONDERWALDT

Yes, that is very similar. What is interesting in sports is that these kinds of techniques have been around for a while and have already been used in many different studies.  Sports are an application in which people want to know about the expert behavior, so they want to understand how to optimize movements and the way they can be learned.

Motion capture is a great tool here because it provides a method for quantitative measurement of movement.  Instead of just watching a video, where you can only look at the image, motion capture allows us to take in movement as numbers on a computer, which means that you can do other kinds of calculations from there.

STRING VISIONS

Statistical analysis, like when you compare a number of first rate players and see what they have in common.  Or what is it that makes them different?

ERWIN SCHOONDERWALDT

Yes, precisely that. Being able to work with numbers instead of just images give you a way to make concrete comparisons based on data, not just qualitative observations.

STRING VISIONS

Have the data you compile using motion capture led you to any theories that could be applied to string playing?

ERWIN SCHOONDERWALDT

Of course. The way I like to see it is that the measurements I take sometimes reveal principles, which you can use as guidelines for playing. I might not go as far as calling it theory yet, but there are certainly principles and guidelines that I have found that are consistent among many players.

STRING VISIONS

Could you mention one or two?

ERWIN SCHOONDERWALDT

One surprising example is in those repetitive bowing patterns, when we play with those typical circular or figure-of-eight bowing patterns that involve coordinating bow changes and string crossings. This kind of figure occurs in the Preludio of Bach’s E Major partita. Within this movement, it seems that you coordinate those two movements—the up and down bow—but at the same time you need to switch to the other string to start a new tone. At first you might think that the string crossing should take place at exactly the same time as the change of the bow. But what you see, if you look closer, is that actually the string crossing takes place a little bit earlier, and the bow change takes place first if you are in full contact with the new string.

STRING VISIONS

That’s true.  That’s a general principle that we all know when we teach; we say you should always prepare the new string level in the upper arm.

ERWIN SCHOONDERWALDT

Yes, but the interesting thing when you are playing fast passages is that the movements of the bow are continuous and more or less sinusoidal. The only way to achieve the earlier timing of the string crossings is then to shift the whole string-crossing movement in time relative to the to-and-fro motion of the bow. That means those two movement components are slightly out of phase, revealing a strong principle of how string crossings and bow changes are combined in this kind of repetitive movements. This Bach Preludio shows a very good example of both of these kinds of bowing patterns. We have the circles there, but you can make the same observation for the figure-of-eight where the string crossing movement is half the frequency of the bow change movement. But it’s still a kind of a nice sinusoid wave but with half the frequency, and that’s why instead of getting a circle, we get a figure-of-eight.

STRING VISIONS

In another study, you showed that there was a strong tendency to slant the bow in a certain direction in order to move the contact point of the bow between the bridge and the fingerboard.

ERWIN SCHOONDERWALDT

That’s also based on the motion capture measurements I did, and it’s another very good example of one of the principles I have found in string playing which can make your life easier if you understand it.  So, in this case I described it as the skewness of bowing being used as a control parameter in order for you to facilitate a change in contact point, as opposed to forcing the contact point to change while keeping the bow at a straight angle.

STRING VISIONS

Meaning, if you want to go to the fingerboard with the bow, you would point the tip in a down bow towards the fingerboard, or if you want to go to the bridge on a down bow, you would point the tip towards the bridge.

ERWIN SCHOONDERWALDT

Yeah, that’s it. It was nice to see that reflected in the measurements, that I could show that the skewness of the bow was very strongly correlated to the changing contact point. That was valid for most of the players; however, there was also one exception, which is very interesting—a very good player, who did not seem to rely on that effect for changing the contact point.  So, there again it might be a good guidance principle, but the truth is it’s always more complex than just applying one single rule.

And that’s also a wonderful thing that I find very fascinating about doing this movement analysis, is that you really get a very clear picture of the complexity of the kind of movements we are dealing with, and it’s a highly complex kind of movement. There are many different details you can recognize. The style of different players is apparent in the data, so we can recognize kinds of personal signatures, which you can then relate to the sounds or the impressions they make when they play.

STRING VISIONS

I love the book or paper that you wrote for your PhD dissertation in Stockholm.  I think it’s a fantastic research project.  But when I read it I have to read it many times to understand it.  I read the whole thing, but to understand it I have to read it many times.  I feel in many ways that helped me clarify certain aspects.  As I said before, when you analyze these things, it’s complex, but in the end, doing it, I think sometimes it’s very simple.

If you think about Michael Jordan, he was amazing because he did things that I’ve never seen any human do under extreme pressure.  He was just fantastic. I remember in one game when they played the Portland Trailblazers, and Jordan—he is not a great three-point shooter— but in the first half alone, he hit 7 three-pointers, and in the end, after the last one, he threw his hands up in the air and looked up at the sky, it’s like, saying, “This is beyond me.”

Of course, he let himself get lost in the moment, and it’s almost like sin where, you know, you let go, you don’t care if you hit it or you don’t care if you miss it, and you just be in the moment.

ERWIN SCHOONDERWALDT

Yeah, that’s probably something which is probably closer to this flow experience.

STRING VISIONS

I think it is, and I think in music we have to go for that.  Many times, we don’t have to make it very complex, and it is not complex for a lot of people.  But if you’re like teaching the instrument or if you want to know more about it, I think we have to really understand how things work.  But of course, many times we don’t want to tell people that.

The more I study and teach, I know more and more about things, but many times I’ll just say one little thing to a student and I’ll never share everything that I know with anybody because nobody needs to know it except for myself.  But for me, reading a lot of your research, it just makes me understand things much better. If I teach a student and they have a lot of things going for them and it’s working perfect, I would never tell them any of those things.  For myself, however, it has enriched my life, and I think it’s fantastic.