This past summer I discussed  the recent comparative study  of abstract art and animal art. Researchers had shown art students pairs of paintings: one by a renowned abstract expressionist and another by a child, a monkey, an ape or an elephant. Researchers then asked the students which painting was better. In 67.5% of the cases art students answered that the painting by an artist is better. Because the figure is above 50%, researchers concluded that their findings “challenge the common claim that abstract expressionist art is indistinguishable from (and no better than) art made by children.” I recalled  another experiment where the subjects compared weights. In 72% of the trials, they perceived a 100-gram weight to be heavier than 96-gram weight, a higher success rate than in the art experiment. I concluded that the difference in artistic weight of abstract artists and animals is less than 4%. I also compared  the results of the art experiment with the outcomes of the games of chess-players of different strengths and concluded that abstract artists are in the same player category with children and animals. My commentary got attention from the media , often critical, and I responded to criticism . However, some of the objections have merit. To compare paintings one should consider many parameters, while to compare weights – only one. More parameters give more reasons for disagreement between judges. In chess, there are no judges at all, since the rules of the game determine its outcome. However, there is a sport where judges decide the results of the competitions: Figure Skating. Now let us look at statistics.
In the Olympics Figure Skating program the skaters compete for four sets of medals in four categories: men, ladies, pairs, and ice dancing. Competition consists of several segments. In men, ladies and pairs categories there are only two segments: short program and free skating. In the ice dancing category there are four segments. The table shows the results  of the 2002 Olympics in Salt Lake City. Under the old 6.0 system , nine judges independently judge each segment. Every judge gives every skater a score which depends on many parameters of performance (including aesthetic parameters, just like in art). Next, they look at all the scores given by a particular judge and rank the skaters from the highest score to the lowest. The skater who gets the highest number of first places wins the segment. Out of the remaining skaters the one who gets the highest number of either first or second places gets the second place and so on. For example, in Ladies Free Skating segment (see Table), Hughes got five first places, more than anyone else, and won the segment. Slutskaya got four first and two second places that is six first/second places and got the second place in the segment. To determine the final placement, they sum the places the skaters get in all segments multiplied by corresponding weight-factors. Free Skate’s weight is 2/3 and the Short Program’s – 1/3. The skater with the lowest total is the winner. If two skaters are tied, the winner is who places higher in the Free Skate. Hughes got the first place in Free Skate and the fourth place in Short Program. Her total is 2/3 + 4/3 = 2. Slutskaya got second places in both segments and her total is 2, same as Hughes’s. Since Hughes placed first in Free Skate, she won the Gold and Slutskaya got the Silver.
Results depend on the arbitrary prescription for breaking ties and arbitrary weights of different segments. If the weights were not 2/3 and 1/3, but 3/5 and 2/5, Slutskaya, not Hughes, would have won the Gold. The aim of this commentary is not to criticize the 6.0 scoring system (they do not even use it today: they replaced it with a far more muddled one). The aim is to show that that the system is robust. When we changed weights, the silver medalist became the gold medalist, not someone from the end of the list. One cannot propose any reasonable modification of the scoring system, which will propel the tenth skater to the first place. The same is true when instead of changing some of the rules we remove the rankings given by some of the judges.
Table 1. Figure Skating results in 2002 Olympics in Salt Lake City. The rightmost columns show the numbers and percentages of judges who placed Gold medalist over Silver medalist, Gold medalist over Bronze medalist and Silver medalist over Bronze medalist. I treated the Pairs category as if Sale and Pelletier got a Silver medal, but discarded the placements given by the disqualified judge.
If you look at the Pairs Free Skate results, you will notice that the placements given by the fourth judge are missing. This is because of the 2002 Olympic figure skating scandal . Initially Sale and Pelletier received the second place in Free Skate and a Silver medal. However, North American news media got angry at this decision. For example, Christine Brennan in her USA Today article “No defense for bad judgment” wrote that she cannot defend the sport of Figure Skating after “Jamie Sale and David Pelletier skated one of the great performances in Olympic history, only to have the gold medal taken from them by five misguided people on the nine-person judging panel.” International Skating Union declared one judge corrupt and discarded her scores. Sale and Pelletier became tied with Berezhnaya and Sikharulidze in the Free Skate segment. Based on that the officials awarded a second set of Gold medals in the Pairs category. One can question this decision because Berezhnaya and Sikharulidze had still won the short program and therefore their total is 1, while Sale and Pelletier’s total is 4/3. To properly award Gold to Sale and Pelletier, one would have to disqualify one more misguided judge. Note by the way, that even if we disqualify all five misguided judges, Berezhnaya and Sikharulidze would fall down only to the second place, not to the twentieth. So much of a scandal.
The aim of the preceding passage is not to ridicule the fight against corruption. On the contrary, the aim is to encourage American news media to uncover corruption in more places. For each segment, I computed the percentage of the judges who placed Gold medalist over Silver medalist (see Table). The average over all 10 segments is 73% . This exceeds the 68% result I started the article with. Thus, the difference between grand masters of abstract art and apes is less than the difference between Gold and Silver medalists. The Figure Skating scandal was about who of the world-class skaters should get Gold and who should get Silver. Though sport critics questioned the Olympics Gold medals of Berezhnaya and Sikharulidze, nobody questioned the two Gold medals they got in two World Championships. In the abstract art experiment, the paintings of renowned artists did not compete with the art of world champions. They competed with paintings any child or monkey can do. Why are the art critics silent? Why is nobody asking how corrupt the judges that decide which of the paintings will cost millions of dollars must be?