Search This Blog

Saturday, February 22, 2014

≠ The sum of its parts

Skating at its finest produces performances that can pass the Bolero test. These are the skates that make life easy for a judge: shower them with 6s (under the old system) or 10s and 3s (these days) and be glad that you were there for something special. Skating, when it's simply very good, produces highly accomplished and sometimes moving performances that also merit very high marks. The problem begins when a competition produces more than one great performance - or conversely, when it produces none. How does one compare skaters and performances whose strengths and weaknesses vary?

Making life easy for all involved



This is a challenge that skating will always face, and I suspect that there may be a better way of addressing it than what the current system allows. What the IJS is designed to do, after all, is to sum up every part of a skater's performance, having assigned each a value, and to see how all these pieces stack up against each other. This isn't inherently wrong, but it's premised on the idea that a skating performance is exactly the sum of its parts. If you agree that this is the right way, or at least the best available way, to evaluate skating, then understanding and accepting most results should not pose a problem. But what if you think that a skater's performance can sometimes be more or less than the sum of its parts? The current system would have us all believe that this is something that cannot and should not be considered. So in 2010 we had Evan Lysacek, The Complete Skater™, and this time it's Adelina Sotnikova, Master Technician and Tactician*. 

Evaluating programs and performances as though they are precisely the sum of their parts ignores a lot of what makes skating special and appealing. It means that a result might come down to who did an extra double at the end of a combination, or held a dance lift a fraction of a second too long, even though the eyeball test might suggest that other factors should weigh more heavily. It also lays on the judges the rather unrealistic task of rating skaters as though they are performing in a vacuum, compared against nothing but rules and bullet points. It's not reasonable or realistic, no matter how well trained judges are, to expect such a thing, and it probably does not fully capture the quality of different performances, either.

I've written before about my belief that the ISU has erred in codifying the current judging system - not by trying to make the judging more objective, but by thinking that it can make it entirely objective and quantifiable. The ISU is not alone; it really is one of the classic blunders. But I'll just reiterate that it's not unreasonable for there to be some subjectivity in judged sports. Not all strike zones are the same size in baseball. Some judges are quicker to call fouls in other sports. There are different cultural and personal preferences in many judged sports. Perhaps the goal shouldn't be to eliminate these entirely, because it's not possible - but to find a way to account for the subjective alongside the objective.

Or maybe the ISU needs to look into conjoint analysis as the future of judging :)

* It should be noted that both Lysacek and Sotnikova skated very well in their Olympic efforts. Just not quite as well as their scores would have us believe.

No comments:

Post a Comment