Why Wine Ratings are Badly Flawed

[Red Bigot]

In more random award news, the top Great Australian Red (Shiraz-cab blends) is the 2006 Yalumba FDR1a Cabernet Shiraz. The best under $20 is the 2008 St Hallett Gamekeepers Shiraz-Cabernet.

Both of these wines failed to win any medal in the NWS.

And FWIW Campbell on the Wine Front scored the Gamekeepers the same as the NWS judges.

[daz]

Can you believe Ringbolt Margaret River Cabernet 2007 is the best Cabernet 2007 and older in Australia? Well it was judged best of all those entered in the prestigious Cabernet Trophy class at the National Wine Show. Most of the red trophies went to Western Australian wines, with Langi Shiraz 2004 taking the best of the Museum classes.
A clear signal to South Australia? Or just too warm during the judging for SA reds to show their best (NWS venue only has evaporative cooling). Randomness in action? Or...?
List of trophies here: http://www.ozwinereview.com/

Best? Doubtful, but I've quite enjoyed the couple of bottles I've had of the Ringbolt. At a quick glance, Sandalford seems to have done well. I've been interested in trying Catching Thieves cab merl for a while now (on discount rotation) but the bloke who manages the local (chain) barn told me he doesn't like the name so doesn't stock it. Go figure..........

"Or....?" Who was the chairperson of judges?

Cheers

daz

[Craig(NZ)]

>>Jeez, ive never read a thread on this topic before!

pass me today's news>>



Craig, couldn't resist!

Yes i was hesitant to pull out that old chestnut!! Its a dusty joke, but a goodie

Sounds extraordinarily boring doesn't it...

Yip im picking not too many wine forums in heaven

[KMP]

The 100 system is obviously something that generates debate. Besides, I've declared war on them (the 100 pt system) and won't rest until it's rotting corpse has been thrown on a mangy old tip, along with Parker to keep it company

Hopefully Rawshack's zeal to kill off the use of 100 points to score wine is not because he thinks that Robert Parker invented it? The earliest use of 100 points to score wine, that I can find, was in a little book called "A Guide to Wine Tasting" by some character called Dan Murphy, published by Sun Books (Melbourne) in 1977. Murphy notes in Chapter 14, in Figures 7 through 13, “a series of score cards which I have used in my business for twenty-five years and which I find useful also for scoring wines at shows.”; which puts an Aussie 25 years in front of a Yank in using 100 points to score wine. Unlike Parker’s simple allocation of 100 points, Murphy’s scoring was different for different wine styles, and would have taken quite a bit of effort to use if scoring the thousands of wines that most professional critics taste in any one year, no doubt one reason why it is not in popular use today.

Honestly, I don't see what all the fuss is about when it comes to giving wine a rating; and lets' face it, wine rating systems whatever they are, have the same flaw, they are one individual's assessment. Anyone who has a serious interest in wine knows that we don't all taste, smell and assess wine identically and so placing your faith in any one critic is a waste of time. BUT being aware of what critics have to say about wine can be useful, certainly helps me sort out what wines or wine regions I might like to sample from. And it certainly beats having to spend the $$$ to taste the hundreds of wines necessary to come to your own decision about any wine variety, region or vintage.

Mike.

[bacchaebabe]

The 100 system is obviously something that generates debate. Besides, I've declared war on them (the 100 pt system) and won't rest until it's rotting corpse has been thrown on a mangy old tip, along with Parker to keep it company

Hopefully Rawshack's zeal to kill off the use of 100 points to score wine is not because he thinks that Robert Parker invented it? The earliest use of 100 points to score wine, that I can find, was in a little book called "A Guide to Wine Tasting" by some character called Dan Murphy, published by Sun Books (Melbourne) in 1977. Murphy notes in Chapter 14, in Figures 7 through 13, “a series of score cards which I have used in my business for twenty-five years and which I find useful also for scoring wines at shows.”; which puts an Aussie 25 years in front of a Yank in using 100 points to score wine. Unlike Parker’s simple allocation of 100 points, Murphy’s scoring was different for different wine styles, and would have taken quite a bit of effort to use if scoring the thousands of wines that most professional critics taste in any one year, no doubt one reason why it is not in popular use today.

Honestly, I don't see what all the fuss is about when it comes to giving wine a rating; and lets' face it, wine rating systems whatever they are, have the same flaw, they are one individual's assessment. Anyone who has a serious interest in wine knows that we don't all taste, smell and assess wine identically and so placing your faith in any one critic is a waste of time. BUT being aware of what critics have to say about wine can be useful, certainly helps me sort out what wines or wine regions I might like to sample from. And it certainly beats having to spend the $$$ to taste the hundreds of wines necessary to come to your own decision about any wine variety, region or vintage.

Mike.

Completely agree, Mike. Not sure why 100 points of scoring is so closely aligned with Parker. I use it myself only for the reason that it gives me a way to compare wines. The verbal description is to emphasise the good and the bad and put some emotion into it but I find the rating useful as a comparison against other wines and to scale them. I personally use it in an absolute sense, not really taking into account VFM or QPR so much. But much like I treat every reviewer of wines, and by that I include every single person who posts here, it just one person's opinon and it's up to each individual to figure out if you have any palate alignment with any given reviewer. Certainly after years here, I know who my palate aligns to and while it's not always 100%, it's usually a pretty good guide.

Brian, why do you keep trying to change the topic? But on that, WTF with the St Hallets Gamekeepers?? I got a case of that at $5 a bottle and even at that QPR, it's not on my list of best under $20 by a long shot.

[daz]

Brian, why do you keep trying to change the topic? But on that, WTF with the St Hallets Gamekeepers?? I got a case of that at $5 a bottle and even at that QPR, it's not on my list of best under $20 by a long shot.

Ian, that's a bloody good price for Gamekeepers, though must admit to having drunk few bottles of it or the Poachers. Are you sure it was the shiraz cab blend and not the grenache shiraz?

Cheers

daz

[Rawshack]

The 100 system is obviously something that generates debate. Besides, I've declared war on them (the 100 pt system) and won't rest until it's rotting corpse has been thrown on a mangy old tip, along with Parker to keep it company

Hopefully Rawshack's zeal to kill off the use of 100 points to score wine is not because he thinks that Robert Parker invented it? The earliest use of 100 points to score wine, that I can find, was in a little book called "A Guide to Wine Tasting" by some character called Dan Murphy, published by Sun Books (Melbourne) in 1977. Murphy notes in Chapter 14, in Figures 7 through 13, “a series of score cards which I have used in my business for twenty-five years and which I find useful also for scoring wines at shows.”; which puts an Aussie 25 years in front of a Yank in using 100 points to score wine. Unlike Parker’s simple allocation of 100 points, Murphy’s scoring was different for different wine styles, and would have taken quite a bit of effort to use if scoring the thousands of wines that most professional critics taste in any one year, no doubt one reason why it is not in popular use today.

Honestly, I don't see what all the fuss is about when it comes to giving wine a rating; and lets' face it, wine rating systems whatever they are, have the same flaw, they are one individual's assessment. Anyone who has a serious interest in wine knows that we don't all taste, smell and assess wine identically and so placing your faith in any one critic is a waste of time. BUT being aware of what critics have to say about wine can be useful, certainly helps me sort out what wines or wine regions I might like to sample from. And it certainly beats having to spend the $$$ to taste the hundreds of wines necessary to come to your own decision about any wine variety, region or vintage.

Mike.

Completely agree, Mike. Not sure why 100 points of scoring is so closely aligned with Parker. .

http://www.erobertparker.com/info/legend.asp

I think that the 'ParkerPoints' registered trademark sort of gives it away

Look, I know that Parker wasn't the inventor of this ridiculous system, but he does seem to have assisted in ensuring it's popularity. I'm sure that there's evidence to support my claim, but I can't be arsed googling this time in the morning. (contradict my earlier claim about us all wanting instant info? Me? Never!)

With reading how the scores are made is interesting, but that might be another discussion for another day

[KMP]

Another point that many argue about the 100 point system is how can any critic be so accurate as to discriminate between individual points on the scale (i.e. 89 versus 88), and that might be a valid argument if the 100 point scale was really a 100 point scale, but its not. As described by Parker it’s a 50 point scale at most as any wine starts out with 50 points just for being liquid in a bottle. But the useful range is even less than that because most users of the scale (e.g. Parker’s Wine Advocate and The Wine Spectator) rarely write up wines that score below 85 (Halliday seems to start his serious consideration at 80 points, not sure about The WineFront). So we are talking about a 15 point system (20 for Halliday). That makes it, in the hands of Parker, even less discriminating than the 20 point scale used on the wine show circuit (most versions of which probably derive from the Davis system which began life in 1959 at UCD).So let’s not give Parker too much credit for his ability to make a 100 point system popular by only making use of the 85-100 point range. Its much more useful to read and taste as broadly as you can, and make your own assessments whether they be mental observations or lengthy tasting notes that use as many scoring options as you can think of.

Mike

[Rawshack]

Another point that many argue about the 100 point system is how can any critic be so accurate as to discriminate between individual points on the scale (i.e. 89 versus 88), and that might be a valid argument if the 100 point scale was really a 100 point scale, but its not. As described by Parker it’s a 50 point scale at most as any wine starts out with 50 points just for being liquid in a bottle. But the useful range is even less than that because most users of the scale (e.g. Parker’s Wine Advocate and The Wine Spectator) rarely write up wines that score below 85 (Halliday seems to start his serious consideration at 80 points, not sure about The WineFront). So we are talking about a 15 point system (20 for Halliday). That makes it, in the hands of Parker, even less discriminating than the 20 point scale used on the wine show circuit (most versions of which probably derive from the Davis system which began life in 1959 at UCD).So let’s not give Parker too much credit for his ability to make a 100 point system popular by only making use of the 85-100 point range. Its much more useful to read and taste as broadly as you can, and make your own assessments whether they be mental observations or lengthy tasting notes that use as many scoring options as you can think of.

Mike

Mike - well written, common sense posts have no place on here. Please revert to feverish rants in future

[bacchaebabe]

Brian, why do you keep trying to change the topic? But on that, WTF with the St Hallets Gamekeepers?? I got a case of that at $5 a bottle and even at that QPR, it's not on my list of best under $20 by a long shot.

Ian, that's a bloody good price for Gamekeepers, though must admit to having drunk few bottles of it or the Poachers. Are you sure it was the shiraz cab blend and not the grenache shiraz?

Cheers

daz

Ian??? It was an insiders sale and I got a heap of stuff including tatachilla partners and other quaffers for $5 a bottle. All off the pallet (literally) in a big warehouse in mascot. Can't remember how I found out about it but obviously some kind soul told me. It's great quaffing at $5 but it's still wouldn't make top 10 in a best under $20 list in my opinion (or my purchases, to be more correct)

[Red Bigot]

Kris, was the Gamekeepers the Shiraz-Cabernet? There is a Shiraz and Shiraz-Grenache etc as well.

[bacchaebabe]

Don't know. I'd have to check. I can't even remember the vintage.

[daz]

Apologies Kris, I sometimes am flummoxed by the pseudonym names and the signoff names to the posts, think I confused yours with N4sir's, had probably been scanning some of his just previous. Haven't had a Tat's Partners for a while but have thought about grabbing a bottle to try when it's been discounted - but not by that much! I'd jump in at that price!

Cheers

daz

[pizzler]

Nothing in the article really surprised me. It just underscored the grain of salt that you should take any rating or award with...a big one. Like a public opinion poll, they are a snapshot of one or a small group of tasters opinions in a slice of time. Bottle variations, the evolution of a bottling over time, bottle temperature, how long the wine breathed, storage, stemware, food, emotions, and anything else that affects the ambiance at the time of the tasting can have a major impact on those opinions. I respect the opinions of wine critics and festival awards (some more than others), but I also respect the body of tasters on CellerTracker, albeit not many of the individual tasters.

[KMP]

More on all those bad, bad, naughty, naughty wine judges here - my advice is avoid all the sniping by Heimoff and co and go straight to the meat and read the last comment by G.M. “Pooch” Pucilowski, the Chief Judge for the California State Fair for the past 25 years and the guy who Prof Hodgson to do the study that was published in the Journal of Wine Economics.

Mike

[Roscoe]

More on all those bad, bad, naughty, naughty wine judges here - my advice is avoid all the sniping by Heimoff and co and go straight to the meat and read the last comment by G.M. “Pooch” Pucilowski, the Chief Judge for the California State Fair for the past 25 years and the guy who Prof Hodgson to do the study that was published in the Journal of Wine Economics.

Mike

Thanks, Mike. "Pooch's" comments are a good read. You gotta respect what he is doing.

[Bick]

Yes top marks for Pooch!

It also occurs to me that this became a thread all about pts systems, when in fact the much more interesting message (for me) was that descriptors for aroma and taste can vary so much not only between judges, but for each judge tasting on separate occasions. It makes me think that the best reviews are those that give a brief summary of those characters that tend to be consistently appreciated (concentration, body, acidity, tannic structure, length, oakiness), and less on the specific aromas and tastes: elderberries, persimmons, lightly peppered Danish salami, damp forest floor, crushed centipedes (not whole ones), slightly wet stones (unlike dry ones, they're so distinct)... etc, etc... Blimey - with half of the descriptors thrown out there in reviews there's no chance of getting either an accurate impression beforehand or recognising the TN when you taste it yourself. Even if you stick to the old favourites, like cherry, blackberry and plum, then its still pretty meaningless, because they're hardly unexpected in pinot, cab and merlot, so they add next to nothing to the review. I'm sure Hugh Johnson, who talks a lot of sense, said much the same thing recently in Decanter.

[Red Bigot]

Yes top marks for Pooch!

It also occurs to me that this became a thread all about pts systems, when in fact the much more interesting message (for me) was that descriptors for aroma and taste can vary so much not only between judges, but for each judge tasting on separate occasions. It makes me think that the best reviews are those that give a brief summary of those characters that tend to be consistently appreciated (concentration, body, acidity, tannic structure, length, oakiness), and less on the specific aromas and tastes: elderberries, persimmons, lightly peppered Danish salami, damp forest floor, crushed centipedes (not whole ones), slightly wet stones (unlike dry ones, they're so distinct)... etc, etc... Blimey - with half of the descriptors thrown out there in reviews there's no chance of getting either an accurate impression beforehand or recognising the TN when you taste it yourself. Even if you stick to the old favourites, like cherry, blackberry and plum, then its still pretty meaningless, because they're hardly unexpected in pinot, cab and merlot, so they add next to nothing to the review. I'm sure Hugh Johnson, who talks a lot of sense, said much the same thing recently in Decanter.

Precisely!
I sometimes enjoy the flow of language in those sort of TN, marvel at the person's vivid imagination - and then ignore all but the basic descriptors.

[bacchaebabe]

Brian and Daz,

checked last night and it was indeed the 06 shiraz grenache gamekeepers.

[daz]

Brian and Daz,

checked last night and it was indeed the 06 shiraz grenache gamekeepers.

I've not tried the shiraz cab Kris but given the comments I've seen of it, your opinion seemed a bit discordant. I agree with you on the grenache shiraz, the couple of bottles I've tried have displayed little to recommend.

Cheers

daz