Tag: critics

Fooling the experts?

An article published last week in the Observer, Wine-tasting: it’s junk science, has proved controversial.  As it covers a subject that interests me greatly – how we perceive, describe and rate wines – I wanted to post about it here immediately, but I was not sure how to start.

Perhaps the best way is to point out that the article itself refers to the work of others, and it represents a fair summary of that work.  If you don’t like the results, read the original studies, and criticise them from a position of knowledge, but do not dismiss them out of hand.  As far as I can tell, with the exception of the widely reported Wisemann study, they are sound pieces of work.

The second thing I would say is that the article’s headline misses the point completely.  Winetasting is not junk science.  It is not any sort of science.  It’s…  well, it’s tasting wine: smelling it, putting it into your mouth, and attending to what you perceive.  Punters, individuals and businesses, do it all the time and base their buying decisions on it.

Tasting itself is not problematic.  The content of the article is about how reliable experts are at describing and rating what they taste.  To be reliable, each individual expert needs to be consistent, and also different experts needs to be able to form a consensus with other people, ideally other experts.  To me it is clear that the studies show there is little reliability.

But that raises another issue: who are these people (who I am calling experts) that are unreliable?  They are different in the various studies.  Judges at shows are typically taken from the wine trade, and can be a mixed bunch.  In the Goldstein study, those who had any form of wines education were deemed to be expert.  And university studies often use oenology students as subjects – these presumably have attended courses on tasting.  Few of the subjects would be Masters of Wine, Master Sommeliers, or even enthusiasts who take part in blind tasting competitions. Were they to be made the subject to research, who knows what the results would be?  Not too dissimilar from the existing studies I suspect.

I do not wish to associate myself with the tone of many of the article comments that glory in the stupidity of the experts and wine snobs alike, but I can see that they have a point.  A lot of the language used to describe wine is rather ridiculous.   Some is designed to elevate the status of the writer rather than to communicate, and other descriptions are simply invented to sell the wine.  Couple that with demonstrable expert reliability problems, and I can absolutely understand the substance of their complaints.

I think the premium end of the wine trade has a big problem in communicating to the general public, in the UK at least.  The answer is not for experts to get defensive about their abilities, and hurl criticism back to punters saying that they are stupid for continuing to buy cheap wine from their supermarkets and that they need to take courses to appreciate wine.  We really do need to move on from trading insults.  In my opinion, as evidenced by his recent book, Eric Asimov gets the tone just about right in his unassuming modesty and willingness to engage with the public.

Update: Just in case you are interested, here are another couple of responses to the article, from Fiona Beckett, Tim Atkin and Victoria Moore.

Reasons not to pay too much attention to tasting notes

Only two reasons for now – if I put my mind to it I am sure I could find more. In summary they are:

  1. There are often variations, sometime very large ones,  in tasting notes from different authorities
  2. There can be quite large variations in one taster’s experience of a wine – well, mine at least, and I am sure I am not alone.

Perhaps the best known example of authorities having widely divergent opinions is the widely publicised spat between Parker and Jancis Robinson over Pavie 2003.  I don’t want to discuss the rights and wrongs of the disagreement here, but do want to emphasise that the two people disagreeing here are hardly johnny-come-lately wine bloggers.

While this case got a lot of attention, it is not at all unusual for well known critics to have very different opinions.  If you read The World of Fine Wine you will see many examples of this in their tastings section.  Spending only a couple of minutes flicking through the latest edition, I find a notes on a couple of Riojas to illustrate my point.  Here we have three tasters:  Tim Atkin (T), Jesús Barquín (J), and Marcel Orford-Williams (M) – maybe not quite in the same league as Parker and Jancis when it comes to authority and influence, but not too shabby.  It is not explicitly stated, but the implication is that each taster is tasting from the same bottle – not that it makes much difference to the reader, who will be drinking a different bottle anyway.  Here are TN extracts that I hope give an impression of the tasters’ opinions, together with their scores:

CVNE Imperial Gran Reserva 1994
T: Mature, feral… gamey… smokey… sweet oak. 15
J: Spicy raw meat… red fruit… fleshy. 17.5
M: Dumb, quite fresh, hard, dry finish. Coarse. 9

Contino Viña del Olivio 2005
T: Firm, chewy… extracted… super ripe… hard to like. 8
J: Balanced structure: acidity, noble tannins… truly excellent. 18
M: Overdone… overextracted and lacking in charm. 15

In case there is any doubt, I am not criticising WoFW or their reviewers.  Quite the reverse in fact – I think it is good that they allow this diversity of opinion to be visible.  But what a range of tasting notes and scores! How is a poor punter meant to interpret this diversity of opinion?  The stock answer is that you should calibrate your palate against the critics and follow those with whom you share tastes.  But I think that is easier said than done.  OK, one might get an impression of a critic’s likes and dislikes, but I doubt very much anyone actually trawls through their own notes and does wine-by-wine comparisons.  I certainly have not enough tasted wines in common with any one critic to do such a thing, though it might be a bit easier to achieve if you are more into high-end clarets.

I would propose that the answer to understanding a wine is to taste it yourself. But it is not quite that simple, and it brings me to my point number 2: variation in my own palate.

A couple of weeks back I attended an informal stand-up tasting at my local wine merchant.  A representative of the producer was pouring, and providing interesting information about the wines, but there was no hard sell. On the back of a small but unhurried tasting sample I bought a bottle of Langmeil Hangin’ Snakes Barossa Shiraz-Viognier 2007.  I didn’t take notes at the tasting, but as I bought the bottle for £12.50 I must have thought it worth 3 or 4 stars.

On getting the wine home, I realised that I had tried it a few months back at with my tasting group, a more leisurely tasting in a home environment.  Checking my notes I was dismayed to see that I was very dismissive of it.  It got 1 star, and I actually used the phrase “cheap and nasty” – ouch! I also thought there was a whiff of hydrogen sulphide about it, and as our hostess had recently acquired a Vinturi aerator we decided to give it a spin (as it were) with this wine.  When served treated and untreated samples blind, I correctly identified the samples and decided that the machine had made the wine drinkable.

So was I now the proud owner of a £12.50 cheap and nasty bottle that could perhaps be improved by a gadget, or was my quick in-store tasting sample to be believed?  I discovered when I took the bottle to Aladdin it was OK, if unspectacular – a grudging 3 stars I guess.  But tasting it at home, both before and after the restaurant trip, it was transformed back to its cheap and nasty mode – a grudging 2 stars.

No, it was not the glass – not entirely at least.  On the Aladdin evening I used the same glass on all occasions.   It was not cork variation, as these bottles were under screwcap. And it didn’t seem to perform consistently worse with or without food.  I do not usually notice such wide variation in my experience of a wine.  But clearly this does happen, and I urge you to bear it in mind when you read any of my tasting notes, or indeed (dare I say it?)  anyone else’s.  I am usually reluctant to to publish notes anyway, and when I do I like to base them on the experience of drinking several bottles on different occasions, though I have offered a few recently based on much more limited experience.

Having said all that, do tasting notes have any value at all?  Yes, I think they do.  For me, my own notes work mainly as memory joggers about previous experiences.  And if my experiences have been inconsistent it is no bad thing to be reminded of that fact too.  But when it comes to the notes of other people I am not so sure.  In my opinion they are mainly useful as a good starting point for a dialog.

Am I going to buy more bottles of  Hangin’ Snakes?  It might be interesting from a scientific point of view, but I am going to save my liver for better stuff.  And if my opinion still counts for anything at all at the end of this article, I would recommend that you take your money elsewhere too.

Trust me, I’m a wine expert

I am fascinated by any scientific study in the field of wine tasting.  So often the results challenge conventional thinking in the wine world and provide much food for thought. Here I shall describe just one piece of research that I think deserves greater recognition.  It is published in The Wine Trials book, and an academic paper that you can download for free.  Do take a look at the paper, but I am not sure I would advise buying the book.  I have the 2008 edition, and most of it is devoted to “100 wine recommendations under $15”.  I enjoyed some of the commentary in the earlier chapters, but I’m a sucker for that sort of thing and even I am not convinced it justifies the purchase price.

The study involved 17 blind tasting events in the USA, held in 2006 and 2007.  There were 506 participants, and 523 wines.  In total 6,175 samples were tasted and rated.   For analysis purposes the participants were classified as expert or non-expert tasters.  Experts were defined as those having had some formal wine training.

The main result is that while the experts’ ratings correlated with price, the non-experts actually preferred cheaper wines.

To give a feel for the magnitude of the effect, the authors give an example of the predictions of one of the models they fitted to the data.  Using the 100 point scale, if there were 2 wines, one costing 10 times as much as the other, experts would rate the expensive bottle seven points higher than the cheaper one, but non-experts would rate it 4 points lower.  The book contains a paragraph of specific results, which I think are useful to put this into perspective: “On the whole, tasters preferred a nine-dollar Beringer Founders’ Estate Cabernet  Sauvignon to a £120 wine from the same grape and the same producer: Beringer Private Reserve Cabernet Sauvignon. They preferred a six-dollar Vinho Verde from Portugal to a £40 Cakebread Chardonnay and a £50 Chassagne-Montrachet 1er Cru from Louis Latour.  And when we concealed the labels and prices of 27 sparkling wines and asked people to rate them, the Dom Pérignon finished 17th – behind 14 sparkling wines that cost less than $15, eight of which cost less than $10.”

There is one very practical lesson to be drawn from this study: if you consider yourself a non-expert you would probably do best ignoring recommendations from experts!

But what is really going on here?  There is probably no single explanation.  A few possibilities spring to mind, but I think the main reason is that the wine trade, from producers to critics, is too inward-looking.  The trade decides amongst themselves what defines a good wine, prices wines accordingly, and then seeks to educate neophytes in the mysteries of the art.  Meanwhile, everyone else feels too intimidated by the whole thing to question the clothes of the emperor.  It seems to me that the negative correlation between ratings and prices indicates that the wine market is organised very strangely.

Does it matter?  Well, yes, it has some very important consequences if sellers of wine are hoping that their punters are readily going to part with more money to get a more enjoyable product.  From my reading of the situation it seems that most drinkers are only likely to trade up if they get so interested in wine that they attend a wine course, or if they decide they need to impress by serving a wine with a prestigious label.

Perhaps that is just the way of the world, but I would be really interested in exploring what non-experts tend to enjoy as a group.  Do they really just prefer sugary pap to Proper Wines?  Or is there a new wine aesthetic waiting to be discovered? Something that future wine makers could aim for with the resources that potential higher prices will yield?

Warts and all

Have you ever wondered why you so rarely seem to see negative reviews of wines?  Or indeed other things?  I am very much aware that the first few reviews on my blog have tended to be positive, so I shall start by answering for myself.

Initially at least, I decided only to write about what I know well, and by and large that is what I have done – certainly my restaurant reviews and longer tasting notes have been for restaurants and wines I am very familiar with.  I didn’t want to be proclaiming judgements based on one meal, or a quick slurp and a spit.  But unfortunately a by-product of that policy is that I have only written in detail about things that I like.  I try to show dedication to my blog, but I draw the line at repeating bad experiences just so I can say with conviction that it was truly bad.

The other reason I might feel tempted to put a positive spin on a wine on a wine that was not great, or more likely say nothing at all, is if I know and like the person that supplied it to me.  I hope using the word “supplied” does not sound too much like having a drug habit fed; I use it to cover both being offered wine by a friend, and being sold wine.  Naturally I do not want to sound ungrateful for freely offered wine, and criticising it in public might be taken as ingratitude, but to a lesser extent I find myself reluctant to criticise wine sold by a merchant I know well.  Though having said that, the careful reader of this blog will find some examples of the latter.

As for other critics… well, I know of at least one who thinks that there are so many bad and mediocre wines it is not worth writing about them, and even listing them it seems.  The consumer of tasting notes is thought only to be interested in hearing about good wines.  I am not at all sure about that.  If someone else has tried a wine and found it to be bad, I would rather not buy it myself to make the discovery independently.  And if no one mentions a wine, how am I meant to know whether it is of poor quality, or simply not assessed?

This is also frustrating for the consumer when reading the results of large wine competitions.  We get to know the wines with trophies, medals and commendations, but how are we to know whether DRC again neglected to submit the requisite number of bottles of La Tâche, or it was judged to be unworthy even of a commendation? And this is where I start to get cynical.  Such a large proportion of wines get medals that to be unclassified is not at all good.  And no producer would want to go to the expensive of entering a competition with the possibility of being slighted like that.  So if the competition published the failures, they would not get get anywhere near the number of contestants and probably the competition would not be viable.

To an extent I think the same applies when writers get sent samples or invited on a jolly – er, sorry, fact finding mission – to a wine producing region.  If there were too many bad reviews the offers of samples and trips would slowly dry up – in general, if not for individual writers.  I am not accusing anyone of professional misconduct here, but I think we have to accept that however hard writers and critics strive to be independent it is hard to be totally objective when your livelihood depends on freebies.  Besides which, as I have noted above, it is really difficult to be critical about a product that is associated with someone you have got to know, like the producer you met on that trip.  I think it is also fair to admit that we as consumers of wine writing get what we pay for.  It is all to easy these to expect to get opinions for free on the net, but those who give their opinions for free need the means to get hold of things to write about.

I am sure that part of the knack of getting the truth from a tasting note or more general review  lies in looking for what has not been said, but that sadly is still a bit like looking for wines that do not have medals.  Did the critic not mention the intensity of flavour because it was insipid, or because he did not think it worthwhile commenting on?  Or perhaps it was not intense, but had an understated elegance?  We will never know.

Another trick in tasting note deconstruction is to look at the score.  I did not realise it until it was explained to me, but apparently a score of below 90 means that a wine is not recommended, while anything you should consider buying will be in the range 90 to 100.  But sadly that now means that some critics are reluctant to give 89 points – so even the points cannot always be used as a coded hint that a wine is under-par.

But you can still get some glimpses of warts.  The blind panel tastings in Decanter for example.  There, often you will find first growths and similar summarily dismissed in favour of more modest wines. What I miss there though is an explanation of the thought processes of the taster.  Of course, better wines need more time to come around, but shouldn’t professionals be able to recognise a young but promising wine from a good stable?