In the 1930s, Pitirim A. Sorokin and J.W. Boldyreff conducted the most interesting experiment1. They told their participants that they are going to play two variations of the same theme by two prominent composers. They declared that musical critics say that the first variation is a masterpiece, while the second is an exaggerated imitation, totally deficient in self-subsistence and beauty. They said that that the aim of the experiment is to see whether laymen agree with the experts. Afterwards, experimenters played the same record twice2. The participants completed a questionnaire which had three possible answers: prefer first record, prefer second record, and no preference.
There was also a 'why' field where people could explain their choice. Out of 863 experimental subjects 501, or 58.1%, agreed with the opinion of the experts and 66, or 7.6% disagreed3. 296 participants, or 34.3%, answered that they do not prefer any of the records, and 80 out of those (this is only 9.3% of all test subjects) wrote in the 'why' field that the two records were identical. The people who preferred the record to itself also made comments in the 'why' field. For example, one of them wrote that the first record meant 'philosophical contemplation of life' and the second record was a 'jerky composition that spoils mood.'
Sorokin and Boldyreff did work at Harvard University and 17 of their test subjects were Harvard students. Out of those 8, or 47%, agreed with the opinion of the experts, 5, or 29% disagreed, and 4, or 24%, indicated no preference. Nobody noticed that the records were identical. These percentages differ from general crowd ones, but some of the differences are not statistically significant since the Harvard sample is small. For example, when only 9.3% notice that the records are identical the probability that no one of 17 people will notice this is 21%.
However, there is one statistically significant difference, and for some reason Sorokin and Boldyreff did not pay attention to it. The percentage of Harvard students who disagreed with the opinion of the experts is almost four times higher than the percentage among the general population. This difference is statistically significant. If the rate of disagreement is 7.6% the probability that 5 or more people out of 17 will disagree is only 1%. In fact the probability that the Harvard disagreement rate is at least twice as high than the general disagreement rate is 95%. It is revealing that Harvard didn't show a higher percentage of people who noticed that the records were identical, but a higher percentage of people who disagreed with the experts on the question of which of the two identical records is better.
Sorokin and Boldyreff wrote that their research helps to understand how people can come to see a great book in a rotten best-seller or a masterpiece in rotten art. I suggest a slightly different perspective. As we all know, rotten best-sellers are read by the uneducated masses, while great books are read by people educated at Harvard. The choice of books made by people educated at Harvard disagrees with the choice made by uneducated masses. I think that the results of Sorokin and Boldyreff suggest that these great books could be of the same quality as rotten bestsellers. Indeed, in a recent experiment4 I discovered that Ivy League and Oxbridge people can’t tell the prose of Charles Dickens from the writings of Edward Bulwer-Lytton, a ridiculed novelist mostly known for a wretched writing contest named after him. In another experiment Ivy League and Oxbridge folks could not tell immortal masterpieces of modern art from the pictures produced by a bored scientist5.
Sorokin and Boldyreff speculated that a large fraction of opinions and judgments of scholars and scientists are of the same nature as the judgments of people in their experiment. I described a couple of such cases in my recent article which tells of the scientists who managed to 'understand' nonsensical hoax lectures.