Show your work.
November 10th, 2008
You have to show your work.
The wine bloggosphere is revisiting the “wine-scoring-is-bad” theme lately. Even the OWC has seen an intense debate over the meaning (or meaninglessness) of numerical wine rating – in which I was very active.
The main arguments in the anti-numbers and anti-objectivity debate are: 1) the 100-point system is meaningless and gives a false sense of accuracy, 2) wine is so subjective that one cannot establish a standardized numerical system and 3) wine is an aesthetic thing, and as a work of art, its quality cannot be captured with a number.
While I do not categorically disagree with all of these three arguments (I disagree with some more than I do with others), there are profound misconceptions underlying the thinking that drives those three arguments.
Rating is meaningless without clear criteria
Whether the author or publication uses three “puffs”, five stars or 10 or 100 points, none have published criteria by which they arrive at a wine’s score. Sure, they’ll tell you that 8 points means “good”, for example, but unless they say what criteria they use to determine a wine is “good” or merits 93 points, I don’t think they are doing anything more than giving their readers some symbol of their personal enjoyment.
But it is not enough to provide a blanket statement describing what one looks for in wine when reviewing. Given the human tendency to take shortcuts and skip steps, a systematic approach to awarding points (or stars or smiley faces, etc) ensures that the ratings arrived at are consistent, reproducible and meaningful.
It goes without saying, though, that it behooves the reviewer to learn to accurately identify aromas and flavors and to develop a meaningful parlance to describe textures, intensities, etc. If a scoring paradigm is to be based on specific characteristics, the reviewer must seek out and identify the wine’s characteristics in order to arrive at a number.
Aesthetics and Judgment
Visual arts, music, poetry all have transcendent elements and effects, for sure. Iambic pentameter, use of myxolydian modes in guitar solos, using small dots of primary colors to render a complex image with a seemingly infinite spectrum of colors or shooting a film in a series of tracking shots all have aesthetic effects. But they are elements of the work which can be identified. Many point to the varied spectrum of personal responses to aesthetic works as the reason why wine cannot be objectively rated for quality. This idea does not hold traction with a logical mind.
The elements of a sensory work (food, wine, art and music) can be readily identified and articulated. They have a basis in science and theory (or rules). Ranking or rating the execution of a particular characteristic or component of a work or product (if that is one’s inclination) is possible and becomes reliable and meaningful when one understands the particular characteristic in its technical context.
One can also judge the execution of the whole work from the perspective of how the individual components come together in the whole finished product. When an experienced and informed reviewer bases their quality judgment on criteria rooted in technical aspects and offers an appraisal of the whole which is not based on their personal enjoyment, that quality rating is more sound.
The problem arises only when one focuses on the individual pleasure response to the whole work. The transcendent aspects of wine, art and music are very personal responses. They are based on our individual histories. They are experience-based memories and connections between two or more unrelated things. Sometimes, they are unexplainable and often unpredictable. They are also separate from the overall technical soundness and quality of a work, product or performance.
While well-written descriptions of the author’s enjoyment (personal response to an aesthetic work) can convey some sense of that transcendent element of the thing being judged, enjoyment should not be a core or focal element of any rating or ranking of wine. It can, however, be offered separately and as a distinct assessment.
I don’t personally know each person who reads my wine reviews. For this reason, I seek ways to tell them about the wine without interjecting the biases of personal preferences or associations which formed in my own mind through my own experiences. My readers should not have to adapt to my associations nor should they have to calibrate their preferences to mine.
On the subject of subjectivity
Personal enjoyment and preferences are absolutely subjective. Enjoyment is also situational. Mood affects enjoyment as do state of health, tasting environment, fatigue and even a where a woman is in her hormonal cycle. These things can also affect the way we perceive different components of wine.
Clark Smith carried out an experiment where he presented subjects with several wines while they listened to different kinds of music. Not surprisingly, certain pieces of music resulted in subjects perceiving some wines as more tannic, for example. There have been times when I did not enjoy or appreciate a piece music as much as I normally would. The music did not sound any different. The notes, harmonies and overtones did not change. The only difference was my response to it and that was directly dependent on my frame of mind at the time.
To use this situational subjectivity and variability of wine perception (and enjoyment) as a trump card against the possibility of objective assessment and rating is simply excusist and belies an unwillingness to attempt to find a viable objective assessment (if not rating) paradigm. If one makes it their job to review wines, they should be systematic in their approach.
There is much a reviewer can do to control their evaluation of wines. Being aware of one’s state of mind and health as well as controlling their tasting environment can go a long way to establishing consistency. This approach may push the comfort level of many, but wine assessors have a responsibility to their audience and not their own egos. It is incumbent upon them to deliver wine descriptions which escape the pull of personal preferences.
What it’s really all about
Whenever a publication – be it print, on-line magazine or a blog – makes descriptions of wines a regular part of their content, they take on a responsibility to their readers. That content is no longer for them or about them. The content exists to benefit the reader/consumer. Any voice, when consistent and persistent enough becomes influential (both with producers and consumers). By endorsing a style of wine they prefer, reviewers who do not seek to objectively convey the character of a wine to their readers are doing those readers a disservice.
Wine writing, describing and rating should be presented in a consistent, clear and unbiased language. The terminology used must be specific and the rating system (if any) should be based on defined criteria. The criteria should be consistently followed without deviation from methodology.
The question of whether all wine critics/raters should adapt the same system is fodder for a separate debate. However, at the very least, those who assess wines should explain to their readers how they arrive at their final judgment and not just offer general terms of what that final judgment means (whether it is represented by a number or other symbol) .
To best serve our readers with the most meaingful information, wine critics should show their work.
Email & Share
November 10th, 2008 at 8:18 pm
Thoughtful post. I think there is a lot of validity to what you say; however, I question the need for scientific methodology in writing about wine. Yes, perhaps if the writing is evaluative – and, particularly, professionally so. However, I think one can establish internal consistency without strict externalized methodology. What is important is, as you mention, to reveal that internal process by being somewhat self-aware. I think with the appropriate caveats a reader can learn from experience how a particular writer approaches his/her subject, and this knowledge can form the basis of thoughtful engagement with the writing. Anyhow, I’ve written more on wine scoring here, if interested: http://just-grapes.blogspot.com/search/label/Thoughts%20on%20Wine%20Tasting and http://just-grapes.blogspot.com/search/label/Wine%20and%20the%20Aesthetic%20Experience
November 11th, 2008 at 10:45 am
Scientific methodology is important in the winery or university laboratory tasting booth. It can be either numeric based on elements of wine character or hedonic. With statistics it gives the wine scientist an tool to use to improve their product or procedures with reproducible results. It is surprising therefore that so few winemakers actually stick to the scientific method. Well, maybe it isn’t surprising, because we all have the grapes and phyiscal tools that we need to consistently make wine that is technically sound in a recipe fashion.
It might be nice if the wine critic had some knowlege of these tools if only for an appreciation for their own fallibility, but it is not to be. If the winemaker can get by with just a whiff and a judgement, why not the critic? If most wine is technically sound, then the rest is just sizzle that eludes numbers or statistics. The sizzle can be whatever you want to make it out to be.
November 11th, 2008 at 11:01 am
You deserve a lot of crdit for sterring the 90-point debate toward science, but I keep wishing you could somehow make it simpler, more direct and more powerful. Can you think of a tasting that would truly test your ideas? If so, perhaps a non-wine magazine/publication/website would cover it.
I till believe that one of the insurmountable problems with ratings is the fact that no critic can replicate his/her blind scores with any reliability whatsoever. If you can disprove or prove that using some real people and real wines, it would be some sort of breakthrough and perhaps make waves beyond the circle of Grape Nuts.
November 24th, 2008 at 11:07 am
[...] Blind tasting is a fun exercise. But it levels the field the wrong way. Wines should be evaluated in the context of cepage and terroir. This is what Appellation America’s Best of Appellation program is about and it is exactly what I argued for in a recent post. [...]