Category Archives: Whisky Resources

Single Malts at the LCBO – October 2015

Well, it’s that time of year again!

After the drought of new single malt releases through the spring and summer, the LCBO is finally starting to stock new expressions for the ramp-up to the holiday season.

Over the last few weeks, I’ve noticed ~40 new single malt expressions on the LCBO website (well, new for this calendar year at least). I’ve just completed an update of my database, and most of these are now included in there. Many of these are higher-end aged expressions, but there are some good (and not-so-good) bargain choices to consider as well. More on that in moment …

Sadly, things aren’t so great on the bourbon front. Here, we continue to lose the mid- and high-end range of popular brands, as US producers adjust their allocations (and cut some international destinations – like Canada – out of their distributions altogether). This de-listing of good quality (and reasonably well-priced) bourbons is a very disturbing trend. See this post on Whisky Buzz for some examples.

But back to happier news – there are lots of new single malts for the Scotch lover to consider. The one sour note here is price – exchange rates do not currently favour the Canadian dollar. And the LCBO has always had some peculiar pricing habits, where certain “popular” brands and/or expressions get walloped with higher-than-typical prices (I’m thinking about you, Balvenie).

When it comes to the higher-end stuff, I will let you browse the database for your own recommendations.  But at the lower-end, there are some interesting new releases to consider, especially in the NAS segment (no age statement).

If you are very budget-conscious, the LCBO is now carrying the Tomatin Legacy for $43.25. That makes it one of the cheapest single malts out there, with a respectable (for the price) metacritic score of 8.25 ± 0.53 on 7 reviews. That is better than the previous entry-level Tomatin 12 yo at $52.25 (7.8 ± 0.63 on 12 reviews).  Keep in mind though that the overall average score for my current whisky database is ~8.5. But again, at $43, that is a simple single malt for less than some blends.

Going up in price, the Jura Brooklyn caught my eye – although my interest soured a little at $79.95. Isle of Jura expressions don’t typically get a lot of love from aficionados, but the flavour descriptions of this one sound interesting. I am only currently tracking one review so far across my metacritic group, although it was fairly positive and above average for that reviewer (60th percentile). One to watch, perhaps, if you have a high risk tolerance.

As always, the Laphroaig Quarter Cask remains a screaming good deal at the LCBO at $72.95 (9.19 ± 0.18 on 14 reviews). But if you want to try something a little different, the new 2015 edition of the Laphroaig Cairdeas is now out ($99.90). Again, it is early for the reviews, but the same reviewer above really liked it (85th percentile score). From the description, it sounds like a slightly fruitier and sweeter version of a typical Laphroaig ~10-12yo (apparently a nod to an earlier style of production). Could be a nice gift under the tree for a classic Laphroaig lover.

Finally, the (new for the LCBO) Kilchoman Loch Gorn gets impressive scores in this heavy-peat class, at 9.12 ± 0.14 on 10 reviews. But is sadly rather highly-priced at $175.95.

For those who don’t like peat (but not so frugal as to go for the Tomatin Legacy), I suppose you could try the new NAS Glenlivet, the laughably-named “Founder’s Reserve” at $52.95. The metacritic score of 8.32 ± 0.19 is based on just 3 reviews, so proceed with caution here. Most scuttle-butt I’ve seen online is that it is inferior to the entry-level 12yo at $56.95 (8.02 ± 0.35 on 15 reviews), when tested head-to-head. So I would easily expect that early Founder’s Reserve score to drop as more detailed reviews come in.

On that note, I’m sorry to say to are likely going to want to skip the new NAS Auchentoshan American Oak at $54.50 (7.75 ± 0.92 on 6 reviews). That is quite a bit lower scoring that the entry-level 12yo at $59.95 (8.33 ± 0.33 on 12 reviews). Indeed, personally I’d recommend you skip all the entry-level NAS in this flavour class and go right to the Auchentoshan 12yo, if you are looking for an inexpensive and unoffensive dram.

As a step-up from there, the newly-released Glenfiddich 14yo Rich Oak sounds interesting, at $65.95 (8.68 ± 0.36 on 6 reviews). That’s quite a score step-up from the entry-level 12yo (8.1 ± 0.26 on 12 reviews), and for only $11 more. Indeed, there are a good number of new Glenfiddichs to consider this year, although most are not as attractive in price.

Aberlour is another one that is typically well-priced at the LCBO, and the new 16yo at $89.95 seems reasonable (8.75 ± 0.19 on 9 reviews). But for $5 more, the A’Bunadh remains your best best in this family, with an overall average across all batches of 9.01 ± 0.22 on 15 reviews. And keep your eyes peeled to see if you can find any old stock of the very well-ranked batch 49 (9.22 ± 0.12 on 5 reviews).

Happy hunting in your LCBO searches!






Whisky in Korea

Selection from the Malt Shop

I’m just back from my second trip to Seoul, South Korea, and had a chance to look into whisky options available there.

Whisky remains a fairly popular drink in Korea, and you will find it on a lot of bar menus. However, the most commonly available choices are generally limited to scotch-style blends, with only a small number of single malts (if any). Prices for the standard scotch fare are generally a little higher than you would pay in North America, but not hugely so. The various expressions of the two common “Korean whisky” brands you will find – Scotch Blue (by Lotte Chilsun) and Windsor (by Diageo) – are typically all blends, sourced from Scottish distilleries for the Korean market.

In terms of selection for purchase, you can be well served by checking out the liquor boutiques in the basement of the major conglomerate department stores (i.e., where the excellent food courts are kept). I perused a couple, but was generally disappointed by the whisky selection and prices (i.e., mainly blends, and rather expensive at that). You do a bit better for wine here, but this is again not exactly a cheap option. Of course, across Seoul there are plenty of small stand-alone liquor stores – but these can be hard to find (and may be difficult to deal with if you are not fluent in Korean).

Your best option for price remains the airport duty free. Unfortunately, the main terminal at Incheon was undergoing renovations when I was there (September 2015), and many of the larger duty free outlets were closed – including the one that has the largest selection of liquor. However, a new large duty free shop recently opened in the Concourse terminal. It had the common whisky items for international duty free, at the usual excellent prices. While again somewhat more heavily biased toward blends than typical, there were a good number of well-known single malt expressions (especially the travel editions). Sadly, there were no Japanese or Taiwanese whiskies present on my traipse through. Also, unlike most duty frees, the whiskies were intentionally scattered across the entire store. This requires you to carefully scan every display, aisle and shelf when looking for products – and interact with a large horde of sales associates at every turn.

Another option is the small but well-organized Malt Shop, in the Gangnam district of Seoul. This store has an excellent collection of international whiskies, as you will able to tell from their website. Be advised however that not everything you see on that site is available for sale (even if it is shown as in stock). For example, while I counted 5 miniature 180mL bottles of the Hibiki 21yo on the shelf, these were all marked “not for sale”. According to the sales clerk, they were part of the owner’s personal collection. And none of the other miniature Japanese bottles shown on the website could be found in the store. That said, most of the full-sized malt whisky bottles listed were available.

The website does not list prices, and I found these to be somewhat variable in-store. Some of the commonly available single malt expressions were quite reasonable – especially the mid-range ones, which were often comparable or even cheaper to what I would pay here at the LCBO (e.g. most of the Balvenies, Highland Parks, etc.). That said, most of the higher-end and entry-level malt whiskies were typically more expensive than you will find in North America. As an aside, the listed shelf prices assume a credit card purchase. If you are paying cash, you may be able to negotiate ~5-10% off these prices.

The inventory was certainly a lot better than what I can find domestically at the LCBO. There were about half-a-dozen expressions available for each of the common Scottish single malt brands (e.g. Ardbeg, Balvenie, Benromach, Dalmore, GlenDronach, Glenlivet, Glenfiddich, Glenmorangie, Glen Moray, Talisker, Tomatin). In some cases, there were even more expressions than I expected to find (e.g., I counted 9 different examples of Arran malts). Some brands only had a couple of expressions available (e.g., Auchentoshan, BenRiach, Bruichladdich, Glenfarclas, Glenrothes, Highland Park, Jura, Springbank, etc.), although that is understandable in some of those cases.

Of course, what I was really looking for was the selection of Japanese and Taiwanese whiskies. 🙂 While there were only two bottles of Kavalan (one Soloist, one ConcertMaster), there were about a dozen or so expressions for each of the Nikka and Suntory lines. Unfortunately, the Nikka ones were largely entry-level expressions (e.g., Super, Gold & Gold, etc.) – including many that I had never even heard of previously. I did however manage to snag the Taketsuru 21yo, which is one I was really looking to find.

Suntory was generally a better mix, with a range from standard Kakubin to the entry-level Yamazaki/Hakushu malts and mid-range Hibikis. Unfortunately, the prices for all the Japanese whiskies were very high, relative to most of the Scottish malts. For example, they wanted ~$300 CAD for the Yamazaki 12yo, ~$400 CAD for the Hibiki 17yo and ~$600 CAD for the 21yo! It’s true that Japanese whisky prices have been rising rapidly lately (and Korea has significant import taxes on Japanese whiskies), but I could typically find those bottles at a quarter of those prices a year ago in Japan. Even the new entry-level Yamazaki NAS “Distiller’s Reserve” was listed at ~$140 CAD. Simply put, Korea is not a place to look for reasonable prices on Asian whiskies – but you can do okay for the Scottish malts.

The Malt Shop, Gangnam, SeoulIn any case, the Malt Shop is definitely worth a visit if you are visiting Seoul and have to some free time. Some of the map links for this store on other blogs are incorrect. Here is a confirmed direct link to google maps, using the store’s address.

It is accessible by public transit, right near the Seonjeongneung subway station. You can access this station off either the yellow Bundang Line (station 214), or the light brown Line 9 (station 927). Once there, take the #4 street exit, and head due south along Seolleung-ro for about 100m – you won’t miss the shop.



Beginner’s Guide to Selecting a Single Malt Whisky

Single Malt whisky guide

Following up on my how to host a whisky tasting article, I thought I’d provide some suggestions of popular, commonly-available, and highly-ranked single malt whiskies in each of the identified flavour Super Clusters.

First thing to do is to familiarize yourself with the characteristics of the individual flavour clusters and super clusters – which you will find described at the bottom of my Flavour Map page. I don’t recommend you get caught up on geographical regions in Scotland (although I will provide classical details below) – it’s far more important to characterize single malts by the flavour characteristics identified through the cluster analysis.

I am going to go through the Super Clusters in the order I recommend when hosting a whisky tasting. That is, starting with the most delicate whiskies and working up to the more complex ones. If you are new to whisky, I also recommend you work your way up the “winey” flavoured whiskies before trying the “smokey/peaty” ones. For those of you more visually-inclined, I’ve posted this commentary as a YouTube video:


Super Custer G-H

  • Dalwhinnie 15 year old is one of the gentler drams, highly ranked in my metacritic database for this super cluster. It’s a Highland whisky with dominant notes of honey and heather/floral aromas. Very easy to drink, and popular with newcomers to single malt whiskies in my house.
  • Glenmorangie 10 year old “original” is perhaps the quintessential delicate whisky that most would be familiar with. Also a Highlander, this is the base spirit that goes into all the more “winey” cask-finished expressions from Glenmorangie (which I personally prefer). But this basic expression does have fans in its own right.

Super Cluster E-F

  • Auchentoshan 12 year old gets a somewhat middling score in my database, but you can’t beat the price – a very good budget whisky. From the Lowland region, it has a delicate base spirit, but has picked up some caramel notes from its time in wood. Fairly dry, it also goes over very well with newcomers to single malts.
  • Redbreast 12 year old is not actually a single malt, but rather an Irish single pot still distillation of malted and unmalted barley. Regardless, it is a good single malt whisky-like dram. Somewhat bolder in flavours and mouthfeel, it is a very highly ranked (and inexpensive) example of this super cluster. Worth venturing across the Irish sea for.

Super Cluster A-B-C

  • BenRiach 12 year old matured in sherry wood is a good introduction to the effects of sherry wood aging on single malts. The base spirit of this Speysider is fairly delicate, so you can really taste the sherry without having overwhelming whisky complexity. A good budget place to start on your heavily “winey” single malt journey.
  • GlenDronach 12 year old “original” is a bolder example of this super cluster, with a stronger range of flavours present (sometimes described as more “meaty” or “savoury”). Technically a Highlander, this one is a lot older than it first appears (as you will see explained in my linked commentary above). Definitely greater complexity than the BenRiach.
  • Aberlour A’Bunadh is a cask strength Speyside whisky (~60% ABV), produced in specific batches (mine is lot 49, but lot 50 is more commonly available now). You will definitely want to add some water to this one, as the full strength effect can be overwhelming. Helps to show off not just the red fruit flavours from sherry wood aging, but the cholocate/mocha richness as well.

Super Cluster I

  • Highland Park 12 year old should probably be in everyone’s whisky cabinet. A good all-rounder from Scotland’s most northerly distillery, on the island of Orkney. A mix of light smokey flavours and sherry, Highland Park is distinctive for its unique lightly peaty characteristics. While the 12 year old won’t win many awards, it illustrates the base characteristics of this distillery well. A poor man’s version of the popular (and much more complex) 18 year old.
  • Talisker 10 year old is a great example of this cluster (especially if sherry is not your thing). Talisker is a peated whisky from the Isle of Skye, and again has some distinctive regional characteristics (described by some as a distinctive sea-air “minerality”). Highly ranked in my metacritic database.
  • Ardmore Traditional Cask gets a somewhat more middling rating in my database, but is great NAS budget choice in this class. Very smokey without being peaty (if that is possible), and more interesting than the similarly priced entry-level Bowmores, in my view.
  • Oban 14 year old is another Highlander like the Dalwhinnie, with similar honey and floral notes. But the Oban is probably more typical of the Highland style, with distinctive smokey notes as well. A bit pricey, which I suspect contributes to its more middling score in my metacritic database. But probably my favourite all-rounder of the four listed here.

Super Cluster J

  • Lagavulin 16 year old is currently my favourite Islay whisky in this class, but it isn’t cheap. A rich flavour explosion, I’ve heard it described as the “depth charge” of whiskies – very popular with experienced drinkers for its complexity and long finish. However, you are likely to smell like a walking ashtray for the rest of the evening (and maybe still the next morning) – so you should warn your significant other before opening a bottle.
  • Laphroaig Quarter Cask (and 10 year old) are two of your best budget Islay offerings in this class. Intensely smokey and peaty, I don’t find there is much else going on here – but some seem to really like these. The QC is better in my view (and the metacritics), and is oddly cheaper here in Ontario – go figure! Great value if you are a fan of smoke/peat.

Of course, those are just starting points for you. Please explore the full Whisky Database for additional options in each flavour cluster.

Expert vs User Reviews – Part II

Following up on my earlier discussion of the online whisky review community on Reddit (“Scotchit”), I have now added a properly-normalized set of Reddit user reviews to my Whisky Database.


As mentioned on that earlier page, I found an unusually high degree of consistency of scoring between Reddit reviewers on some whiskies, and evidence of systematic biases that differed from the independent expert reviewers. Scoring methods were also a lot more variable among the user reviews, although this can be partially corrected for by a proper normalization (as long as scoring remains consistent and at least somewhat normally-distributed for each reviewer).

My goal was to find Reddit reviewers who could potentially meet the level of the expert reviewer selection used here. As such, I started by filtering only those reviewers who have performed a similar minimum number of reviews as the current experts in my Whisky Database (in order to ensure equivalent status for normalization). This meant excluding any Reddit reviewer with less than 55 reviews of the current ~400 whiskies in my database. As you imagine, this restricted the number of potential candidates to only the most prolific Reddit reviewers: 15 in this case.

Upon examining the scores of these generally top-ranked reviewers, I identified 6 as having potential inconsistency issues in scoring. One common issue was a non-Gaussian distribution (e.g., a much longer “tail” of low scoring whiskies than high). I was able to account for this in the final analysis by slightly adjusting the normalization method at the low-end.

Of potential concern was inconsistent reviewing, where two products of similar flavour profile, price and typical mean expert scores were given widely divergent scores. Only a small number of reviewers should issues here, but some examples include the Aberlour 12yo non-chill-filtered compared to the Balvenie DoubleWood 12yo, the Coal Ila 12yo compared to the Ardmore Traditional Cask, and the Glenfiddich 18yo compared to the Dalmore 12yo.  I found reviewers who placed those exact pairings at the extreme ends of their complete review catalogue (i.e., ranked among the best and worst of all whiskies reviewed by that individual).

To be clear, this is not really a problem when perusing the Reddit site. As long as you are looking at whiskies within a given flavour cluster, you are likely still getting a clear relative rank from these reviewers. It is just when trying to assemble a consistent ranking across all flavour classes of whiskies that inconsistent review scores for a given reviewer becomes a potential issue. As explained in my article discussing how the metacritic score is created, scoring is simply a way to establish a personal rank for each reviewer.

Fortunately, these instances were fairly rare, even for the reviewers in question. In most cases, it was the low-ranked whisky that was disproportionately un-favoured for some reason. If significantly discordant with the rest of the database, these could be accounted for in the normalization by excluding a small number of statistically-defined outliers (using the standards described below).

Correlation of Reddit Reviewers

Independence of review appears to be lower among the Reddit reviewers than among the expert reviewer panel used here. Reddit reviewers often reference the scores and comments of other users in their own reviews. This tends to lead to some harmonization of scoring, perpetuating dominant views on a number of whiskies. Indeed, the variance on many of the well-known (and heavily reviewed) expressions was lower for normalized Reddit reviewers than the expert reviewer panel. Also, the average of all significant correlation pairings across the 15 Reddit reviewers was higher than among the expert review panel (r=0.60 vs 0.40). Interestingly, the one Reddit reviewer who seemed the most independent from the others (r=0.37 on average to the others) correlated the closest with my exiting Meta-critic score before integration (r=0.72).

I also noticed a strong correlation in the selection of whiskies reviewed among Reddit reviewers – which was again much higher than among my independent expert reviewers. This was initially surprising, given the wide geographic distribution of Reddit users. But on further examination, I discovered that lead Reddit reviewers typically share samples with one another through trades and swaps. This can of course further reduce the independence of the individual reviews.

As a result of this analysis, I decided to combine these 15 Reddit reviewers into one properly normalized reviewer category for my Whisky Database (i.e., a single combined “Reddit Reviewer” category).  [Revised, See Update later in this review]

Normalization Method for Reddit Reviewers

For this analysis, each Reddit reviewer was individually normalized to the overall population mean and standard deviation (SD) of my current expert review panel. The normalized scores for these individual Reddit reviewers were then averaged across each individual whisky they had in common, to create the combined Reddit Reviewer category. On average, there were n=5 individual Reddit reviewers per whisky. As expected, the SD for the Reddit group of reviewers was lower on average than among my current expert panel.

To deal with any inconsistent scoring patterns, I used fairly stringent criteria to isolate and remove outlying scores. To be considered an outlier, the individual normalized Reddit reviewer score had to differ from the average Reddit reviewer score for that whisky by more than 2 SD units, AND had to exceed the existing Meta-critic score by more than 3 SD units. In cases where only one Reddit reviewer score was available, exclusion was based solely on the 3 SD unit criteria from the existing Meta-Critic mean score. This resulted, on average, in 0.8 outlier scores being removed from each Reddit reviewer (i.e., less than one outlier per reviewer).

The combined group of top Reddit Reviewers was then treated as a single reviewer category in my database. The combined Reddit score was then integrated with the other expert reviews for all whiskies in common (~200 whiskies in my database). The second pass normalization was performed in the same manner as for each individual expert reviewer described previously.

Comparison of the Reddit Reviewers to the overall Metacritic Score

Now that this category of top Reddit user reviews is properly integrated into my Whisky Database, it is interesting to compare how the Reddit scores compare to the other experts – to see if the general trends noted earlier in Part I persist.

For the comparisons below, I am comparing the combined Reddit Reviewer scores to the revised total Meta-critic scores (which now includes this Reddit group as a single reviewer category). I will be reporting any interesting Reddit reviewer differences in terms of Standard Deviation (SD) units of that group from the overall mean.

I am happy to report that the integrated Reddit scores do not show the pattern of unusually low ranking for international whiskies, as noted previously for the broader Reddit group (i.e., these top reviewers are commensurate with the other expert reviewers here).

For the Rye category, the overall distribution of scores was not that different. There was a trend for Reddit reviewers to rank Canadian ryes lower than American ryes, but the numbers are too low to draw any significant inferences.

The Bourbon category similarly shows no consistent difference between the Reddit reviewers and the other reviewers in my database. The Jack Daniels’ brand of whiskies seems somewhat less popular on Reddit, however, compared to the overall Meta-critic score (Gentleman Jack -2.2 SD, Jack Daniels No.7 -1.1 SD, Jack Daniels Single Barrel -0.5 SD).

The blended Scotch whisky category was scored lower overall by the Reddit reviewers compared to the expert reviewers – consistent with the earlier observation (i.e., almost all Scotch blends received a lower Reddit score than the overall Meta-critic score). Only a couple of blends stood out as being equivalently ranked by both the top Reddit reviewers and the other reviewers – the most notable being Té Bheag (pronounced CHEY-vek). Incidentally, this happens to be one of the highest ranking Scotch blends in my database. To be clear: Scotch blends get consistently lower ranks than single malts by virtually all reviewers – it’s just the absolute scoring of the normalized Reddit reviewers that is particularly lower than the others.

The single malt whiskies showed some noticeable examples of divergence between the Reddit reviewers and the overall Meta-critic scores. The clearest example of a brand that was consistently ranked lower by the Reddit group was the new Macallan “color” series (Gold -2.4 SD, Amber -1.4 SD, and Sienna -0.5 SD). To a lesser extent, a similar pattern was observed for some of the cask-finished Glenmorangie expressions (Nectar D’Or -1.1 SD, Lasanta -1.0 SD), and the entry-level Bowmore (12yo -1.4 SD, 15yo -1.3 SD), and Ardmore (Traditional Cask -2.1 SD) expressions.

Similarly, some brands got consistently higher scores from the top Reddit reviewers – most notably Aberlour (A’Bunadh +2.0 SD, 12yo NCF +1.8 SD, 12yo double-cask +1.6 SD, 10yo +0.4 SD). Again, to a lesser extent, other seemingly popular Reddit choices were Glenfarclas (105 NAS +1.1 SD, 17yo +0.9 SD, 12yo +0.7 SD, 10yo +0.6 SD) and Glen Garioch (Founder’s Reserve +1.3 SD, 12yo +0.8 SD, 1995 +0.4 SD).

Note that both Aberlour and Glenfarclas are generally in the heavily “winey” end of the flavour clusters, just like the relatively unpopular Macallans (and Glenmorangies). I suspect part of the issue may be the perceived value-for-money in this “winey” category. Macallan is considered especially expensive for the quality, and the new NAS “color” series are generally regarded as lower quality by most critics (and even more so by Reddit reviewers). In contrast, Aberlour remains relatively low cost for the (high) perceived quality.

In any case, those were among the most extreme examples. On most everything else, there is little obvious difference between the normalized top Reddit reviewers and the other expert panel members. Properly normalized in this fashion, they provide a useful addition to the Meta-critic database. I am happy to welcome their contribution!

UPDATE July 22, 2016:

In the year since this analysis was published, I have continued to expand my analysis of Reddit whisky reviews.  I now track over 30 Redditors, across my entire whisky database, properly normalized on a per-individual basis.

While many of the observations above remain, this larger dataset has allowed me to explore reddit reviewing in more detail.  Through correlation analyses, I have been able to refine subtypes of reviewers on the site.

Specifically, there is a core set of reviewers who show very high inter-reviewer correlations.  This group, as a whole, correlates reasonably well with the Meta-Critic score, but is really defined by how consistent they are to one another.  Many of the high-profile, prolific reviewers fall into this group. All the associations noted above apply to this group, and are strongly present (e.g., they score American whiskies consistently higher than the Meta-Critic, and Canadian whiskies consistently lower).

A second group of reviewers show relatively poor correlations to each other, the main reddit group above, and the Meta-Critic score. On closer examination however, the main reason for this discrepancy is greater individual extremes in scoring on specific whiskies or subtypes of whisky.  When properly normalized and integrated, this group demonstrates a similar whisky bias to the first group (although somewhat less pronounced, and with greater inter-reviewer variability). A number of high-profile reviewers fall into this second group.

The third group (which is the smallest of the three) is a subset of reviewers who correlate better with the Meta-Critic score than they do the two groups above.  This group appears to show similar biases to the larger catalog of expert reviewers, and not the specific cohort of reddit reviewers.

As a result of these analyses, I have expanded the contribution of reddit scores to my database by adding the average scores for each group above.  Thus, instead of having a single composite score for all of reddit on each whisky (properly normalized and fully integrated), I now track 3 separate reddit reviewer groups (each normalized and integrated for that specific group).

I believe this gives greater proportionality to the database, encompassing both the relative number of reddit reviews, and their enhanced internal consistency.


Expert vs User Reviews – Part I

Wine barrels

As discussed on my Biases and Limitations page, I have chosen to use only established reviewers – with an extensive range of individual whiskies reviewed – when building the Whisky Database here. Please see my Reviewer Selection page for the criteria used in selecting these reviewers.

But there are a number of active online whisky communities that have member reviews, and it is worth exploring how these may relate to the properly-normalized expert panel developed here. In this commentary, I am going to explore correlations of the Whisky Database to the Reddit Scotch Whisky subgroup (“Scotchit”).

My goal here is simply to see whether or not it is worthwhile to try to incorporate this user community into my expert panel. I am personally a big fan of discussion forums, where newcomers and experts can rub shoulders and shares experiences.

The Reddit Scotchit Review Archive

This Scotchit user group meets many of my established reviewer criteria, including being very active in the last few years with openly available reviews. While the main Scotchit site can be a bit daunting to navigate, you can find the full open-access review archive (with over 13,000 reviews as of July 2014, including “community reviews”) – as well as several attempts at quantitative analysis and summary of the results.

The main challenge is that individual Scotchit user reviews can vary widely in quality, experience and consistency. Scoring is also hugely variable (i.e., some members use the full range from 0-100, whereas others use the more common restricted higher range). Ideally, a proper normalization should be performed for each reviewer, but this poses considerable technical and logistical challenges for the massive review archive dataset. User Dworgi has created a user review normalizer program, but I couldn’t get it to work with the current review archive.

On that front, I should point out that while they have done an impressive job of maintaining the Scotchit review archive, it is still a community project using automated review-catching bots. As a result, there are a certain number of errors in the database. The most significant of these are erroneous scores (likely due to the reviewer mentioning several scores in his/her review, with the automated script having trouble finding the final score). There are also structural problems with the complete dataset – for example, several hundred entries currently have missing columns or transposed columns. There are also many more cases where the same expression is listed under different titles. So if you plan to work with this archive, you will still need to do your own manual quality control checks and data curation.

Given these issues (which are generally well appreciated on the site), it is recognized that some filtering restriction of the archive is required to meaningfully interpret any summary results. One approach is to restrict to only those reviewers that meet a certain minimum number of published individual reviews (e.g., those who have done 50 or more), and to ignore community reviews. Another is to set a minimum number of user reviews required for each whisky before considering it in an analysis (e.g., 10 reviewers, as done here for an analysis by Dworgi for data up to the end of 2013). Another option is to also restrict reviews to those that meet a minimum score cut-off (e.g., neglecting reviews that score <50 out of 100). Charles Vaughn has a good interactive graphing tool using the same dataset as Dworgi, where you can dynamically adjust these cut-off values yourself on the Overview tab and see how it affects the results. This is a good tool to help you calibrate you understanding of the dataset (although it is limited in time to an early summary set).

Again, all these restrictions are done in order to try and help compensate for the wide variations in scoring, given the lack of normalization. Given my experience, I’m not sanguine about the success of these methods – as demonstrated on this site, you really need to properly normalize each reviewer’s scoring if you are to meaningfully integrate. However, given the daunting size of the Reddit reviewer database, these simple filtering approaches are understandable – and are better than nothing. It is at least worthwhile to see if the filtering methods suggest any meaningful trends that could be followed up  with a more detailed analysis.

As an aside, one potential advantage to having a very large dataset of user ratings is the possibility of using a proper Bayesian estimator. Popular in estimation/decision theory, a Bayes estimator is used to compensate for when only a small number of ratings are available on any given item in a much larger dataset (i.e., what is known as posterior expected loss). It works nicely across extremely large datasets that have highly variable numbers of reviews (e.g., the Internet Movie Database apparently uses a Bayesian estimator). Items with a low number of reviews are given lower weighing in the analysis. Once the number of ratings reaches and surpasses a defined threshold, the confidence of the average rating surpasses the confidence of the prior knowledge, and the weighed Bayesian rating essentially becomes a regular average. Of course, any biases in the underlying dataset would still be confounding (see below), but this would definitely be something to consider if you want to mine the Scotchit database further.

Comparative Analysis

For this first pass analysis, I have pulled out of their current public review archive (as of July 2014) all reviews for whiskies in my dataset. This yielded >200 whiskies in common, with almost 6000 individual Scotchit reviews. A quick descriptive-statistic examination of the raw data illustrates that single malt-like and bourbon-like whiskies get generally equivalent average scores across Scotchit reviewers, but that scotch blend-like and rye-like whiskies get significantly lower scores on average. While this trend is apparent among my expert review panel as well, it is noticeably more pronounced in the Scotchit user archive. This feature is well noted on the site – i.e., many seem aware that blends (and other perceived lower quality whisky categories) are particularly devalued by the members.

At a more granular level, I note that the international malt whisky subset of my database are scored lowered by the Scotchit users on average, compared to the Scottish single malts. In contrast, the expert panel used here rates these international whiskies higher than the single-malt group average. Further, I note that Canadian rye whiskies get lower scores on average in the Scotchit review archive than the American rye whiskies – whereas those two sub-categories of rye get equivalent scores among my expert panel. While relative numbers are low in these last two cases, it does suggest that the underlying biases may be different between the expert panel and Scotchit users for international products.

To explore relationships between our datasets in more detail, I have applied the same filtering method used by Scotchit users themselves when depicting their own data. For this analysis, I have used moderately stringent criteria, excluding all whiskies with <10 reviews, and any individual review score <60. This reduces the dataset to ~5400 Scotchit reviews, across ~160 whiskies in common. Again, I would prefer to use proper normalization of each of their reviewers, but the automated tool does not currently seem to be functioning.

A few observations of this restricted dataset. For one, the variation across Scotchit user reviews is much higher than across my expert review panel, on a per-review basis (even before normalization was applied to my dataset). And even after consolidating to an average score for each whisky, the variation within each reviewer group is again much higher for the Scotchit dataset. These results are not surprising given the wide variation in how scoring is applied by Scotchit users (and despite the attempt at filtering the results).

Correlation to the Meta-Critic Scores

Let’s see how a correlation of the average Scotchit score for each whisky compares to my meta-critic dataset. For this depiction, I have broken out the whiskies by category (single malt-like, scotch blend-like, bourbon-like and rye-like). Further, I have identified the flavour cluster categories for the single malt style whiskies. See my How to Read the Database for an explanation of these terms.

Correlation of to Reddit Scotchit

The overall correlation to my meta-critic score is reasonable (although lower than the average expert reviewer to the meta-critic score). You can also see that the variation increases as we move to whiskies with lower scores, as you might expect given the significant variation in how the Scotchit user base applies scores. Note that this variation is much higher than that seen in my expert reviewers (and discussed here).

Looking more closely, you can also see how the blended whiskies do indeed score relatively lower for the Scotchit users (i.e., virtually all these whiskies are below the best fit line, indicating relatively lower Scotchit scores). There are no obvious group differences among the other classes or clusters, suggesting the Scotchit users share a similar bias to my expert review panel of favouring complex whiskies over delicate ones, and heavily-smoky over lightly-smoky (in general terms).

That said, there are a number of individual whiskies results that are interesting (i.e., cases where the two reviewer bases obtain significantly different results). One example is for the less heavily-peated Highland and Islay whiskies (e.g., Bowmore and Ardmore). The entry level expressions of these brands tend to get average-to-slightly-above ratings in my expert panel, compared to consistently below average ratings from the Scotchit user base. I am not clear as to the reason for this difference, but it may reflect how user reviewers tend to apply a wider scoring scale within a given class of product (i.e., while they rate heavily-peated whiskies equivalently to the expert reviews, they disproportionately rank lighter-peat whiskies lower).

There are also individual whiskies where the Scotchit rankings seem unusual. One example is the Glenmorangie Nectar D’Or. The other Glenmorangie expressions rank in similar ways between the Scotchit and meta-critic reviewers (i.e., Lasanta < 10 yr Original < Quinta Ruban < Signet). The Nectar D’Or gets an overall average score among Scotchit users (placing it in the middle of the pack), but a consistently high score among the meta-critic reviewers (i.e., equivalent to the Signet). A possible explanation for this is that the Nectar D’Or is frequently cited in some of the popular Scotchit analyses as an example of an “average” scoring whisky (going back several years now). This may thus be influencing members to consistently rank it that way, based on earlier assessments (i.e., a trend towards consistency over time).

Wrapping Up

Taken together, this analysis suggests that may be some specific and general differences in the underlying scoring approach taken by Scotchit users and the experts used here in the meta-critic score. In particular, Scotchit users seem more critical and relatively negative toward perceived lower quality whiskies (e.g., blends) than the expert reviewers. Similarly, the users may have a relative bias in favour of UK single malts and American bourbons/ryes compared to other international jurisdictions of similar products, relative to the expert panel. Note that I am not saying the expert panel is better or worse in this regard – just that their relative systemic biases may be different (and thus difficult to integrate across users and reviewers).

There are also definite differences in how scoring is applied to whiskies, with a lot more variation among Scotchit users. However, it may be possible to correct for this with a proper normalization for each reviewer. And a proper Bayesian estimator could be used to adjust for cases where there is a low number of reviews. To date however, it seems that simpler filtering approaches have been used for most analyses of the Scotchit archive.

An underlying question to explore in more detail is the relative level of independence of reviewers. Even the expert reviewers used here are bound to be influenced to some degree (likely a variable degree) by the reviews of other experts. There are indications in the Scotchit analyses that this effect is more pronounced among the members of the user group, especially in regards to certain specific whiskies and defined clusters of whiskies. This may pose an insurmountable problem in equating expert reviews (where distinctiveness of review – within overall consistency – is highly prized) and community user reviews (where consensus may be highly prized by some, and extreme/inconsistent positions valued by others). Again, the relative value of the meta-critic analysis here is that it is drawn from samples that are as independent as possible, while striving for internal consistency (both important criteria for inferential statistics).

At the end of the day, it would not be appropriate to try and incorporate any simple summary of the Scotchit archive into the expert meta-critic score. However, there may be individual reviewers from Scotchit who have similar characteristics to the experts used here (i.e., similar relative biases and levels of independence). Indeed, there is one member that is common to both groups (the Whiskey Jug). I will continue to explore the individual reviewers in more detail, to see if there are any others that may be relevant for inclusion in the current meta-critic group.

And of course, none of this should get in your way of joining and participating in any user community you feel a connection with. They can be a great place to explore ideas with people of similar interest. 🙂

UPDATE: I proceeded further through the dataset, and done proper statistical normalizations on a number of Reddit reviewers. Discussed further here.



Whisky Volume in Ontario

display rack of whisky bottles

Summer is apparently a slow time for whisky sales in Ontario – at least judging by all the marketing push at the LCBO for everything but whisky these days.

I look forward to the lead-up to Christmas, when the LCBO stocks a much wider array of mid-range and higher-end whiskies (especially single malts). Right now, it is the high volume, low-end whiskies that seem to dominate on the shelves. With Canada Day almost here, I thought I would do an analysis of the summer product distribution of Canada’s largest distillers (thanks to the helpful LCBO app).

What I’ve done is to search the LCBO app and website for all “budget” Canadian whiskies (i.e., in the $25-30 range). From this, I developed a list of all producers/distillers currently offering entry-level products (as a proxy for the largest distillers). I then tracked the LCBO inventory for the entire brand and expression range of the whiskies these distillers produce, in all bottle sizes. There were half a dozen distillers with less than 6000 liters of product on inventory at the LCBO, which I excluded from the further analysis (you’ll see why in a moment).

At the end of the day, this gives me a good dataset of the full pre-Canada Day LCBO inventory of all the products made by the largest Canadian whisky producers. The full searchable data table is available at the bottom of this post.

Before I get into the top-line observations from this whole dataset, there is one interesting feature to note in a subset of whiskies. Most of the entry-level expressions of these major distillers are available in a range of bottle sizes (i.e., from 200mL up to 3000mL). For all whisky brands where multiple bottles sizes were available, here is the distribution of the total current LCBO inventory (combined from all producers), as of June 28, 2015:

LCBO-bottlesAs you can see, the 750mL and 1140mL sizes are the clear “sweet spot” for the production and sale of budget whiskies (not surprisingly). What may be a little surprising is the absolute number of bottles in inventory. Again, the above chart is only for expressions that are released in multiple bottle sizes (which are typically just the base expression of each producer).

So what are the key observations from the whole set?  Let’s start with a ranking by production levels, for each identified distiller on the LCBO site. Note that these distillers may in fact be distributors (and some are owned by the same conglomerates). As such, I will also provide the names of the specific brands they produce, since that is what you are likely to recognize. Here they are, in order of total inventory of all whisky products (in liters) at the LCBO right now:

  1. Corby Distilleries Ltd (Wiser’s, Hiram Walker, Royal Reserve, Lot 40) – 148007 liters
  2. The Crown Royal Distilling Co. (Crown Royal) – 105383 liters
  3. Forty Creek Distillery (Forty Creek, Canada Gold) – 96603 liters
  4. Canadian Club Whisky Company (Canadian Club) – 79156 liters
  5. Schenley (Seagram’s, Black Velvet, Golden Wedding) – 52258 liters
  6. William Grant & Sons (Gibson’s) – 43819 liters
  7. Carrington Distillers (Alberta Premium, Alberta Springs) – 41460 liters

The relative rankings aren’t too surprising to me, especially with the Wiser’s juggernaut in Canada. I’ve noticed a lot people buying Crown Royal, so that makes sense as second place. But Forty Creek is certainly quite the success story, given that they have only been around for a short period of time. The Alberta distillers do remarkably well in Ontario as well, it seems.

The absolute volume of whisky in Ontario is a bit of an eye opener for me. Just focusing on these major distilleries, that’s over 566,000 liters of Canadian whisky currently sitting on LCBO shelves (at the time of year when whisky sales seem to be at their lowest).

What percentage of those whiskies are mid-range or higher? Very little. If you crunch the numbers, it turns out that (by volume) 96.1% of the output of these Canadian distilleries goes into <$30 whiskies. That means that less than 4% of their output (based on current LCBO inventories) could even have a shot at being called a “sipping whisky”. The vast majority is clearly designed for mixed drinks, as entry-level product.

To add insult to whisky-snob pride, want to guess how much of this total output is geared toward flavoured whisky products? According to the dataset, 8% of the above are flavoured whisky drinks. And that is likely an underestimate of the true market share, since there are other popular flavoured whisky drinks that are not being captured in this analysis. For example, if we add the popular cinnamon-flavoured shooter “Fireball” into the dataset, the overall proportion of flavoured whisky rises to 10% of the new total.

For the higher-end whisky enthusiast, these results can only be described as … sobering.

EDIT: As an aside, here is an interesting article from outlining their challenges with the LCBO.

Below is the full table, in number of bottles sitting in inventory for the LCBO on June 28, 2015.

Standard Pice (750mL)
# 200mL
# 375mL
# 750mL
# 1140mL
# 1750mL
# 3000mL
Total Litres
Carrington DistillersAlberta Premium$25103901389088776426664532899
Carrington DistillersAlberta Premium Dark Horse$3133512513
Carrington DistillersAlberta Springs$251272416816048
Canadian Club Whisky CompanyCanadian Club (Premium)$2614118138371571513505717986150341
Canadian Club Whisky CompanyCanadian Club Reserve$2730822312
Canadian Club Whisky CompanyCanadian Club Clasic$2813461447183793161612423
Canadian Club Whisky CompanyCanadian Club Maple$2743013226
Canadian Club Whisky CompanyCanadian Club Rye$2775871642156510301
Canadian Club Whisky CompanyCanadian Club Sherry$33738554
The Crown Royal Distilling Co.Crown Royal$2995731220132400714881117075344
The Crown Royal Distilling Co.Crown Royal Apple$301772813296
The Crown Royal Distilling Co.Crown Royal Maple$3052263920
The Crown Royal Distilling Co.Crown Royal Black$33716525218248
The Crown Royal Distilling Co.Crown Royal Limited Edition$4022751706
The Crown Royal Distilling Co.Crown Royal Special Reserve$551200900
The Crown Royal Distilling Co.Crown Royal Monarch 75th$6021461610
The Crown Royal Distilling Co.Crown Royal Extra Rare Blue$180479359
Forty Creek DistilleryCanada Gold$2511320721637449
Forty Creek DistilleryForty Creek Barrel Select$2718743141971922513292562748491
Forty Creek DistilleryForty Creek Spiked Honey$2892346926
Forty Creek DistilleryForty Creek Copper Pot$30106966891289720947
Forty Creek DistilleryForty Creek Cream$298057404010648
Forty Creek DistilleryForty Creek Double Barrel$6017211291
Forty Creek DistilleryForty Creek Confederation Oak$701133850
William Grant & SonsGibson's Sterling$2766967144432020726
William Grant & SonsGibson's 12yo$30512981557202362922601
William Grant & SonsGibson's 18yo$75656492
SchenleySeagram's 83$2537273729273511833
SchenleySeagram's Five Star$2554
SchenleySeagram's VO$2555166696327117495
SchenleyBlack Velvet$2424121304002
SchenleyBlack Velvet Toasted Caramel$2726561992
SchenleyGolden Wedding$251443293552351213447
SchenleySchenley's OFC$259319313485
Corby Distilleries LtdRoyal Reserve$2555085754317316243
Corby Distilleries LtdHiram Walker's Special Old$256496740655004924407420944
Corby Distilleries LtdLot 40$4017131953
Corby Distilleries LtdWiser's Special Blend$25121611264010280641736989
Corby Distilleries LtdWiser's Deluxe$2716926164151647621014851155662416
Corby Distilleries LtdWiser's Torched Toffee$281162872
Corby Distilleries LtdWiser's Spiced Vanilla$28318436723948
Corby Distilleries LtdWiser's Small Batch$3037742831
Corby Distilleries LtdWiser's Legacy$50789592
Corby Distilleries LtdWiser's 18yo$701007755
Corby Distilleries LtdWiser's Red Letter$100619464

Further Reading

Canadan whisky on ice

It can be hard to find reliable information on whisky out there. In addition to the specialist websites that I have linked to throughout this site (and on my reviewers page), here are some worthwhile reads to consider (in no particular order)

Dave Broom’s The World Atlas of Whisky is a classic. Frequently reprinted (and updated), this is the best “coffee table” whisky book I know. Gorgeously illustrated, with tons of background information on distilleries and individual whiskies. Provides recommendations on whiskies to try next (but these seem very idiosyncratic and personal, and not based on any objective analysis). Same goes for the flavour mapping, as discussed here. Still, as long as you not expecting an explanation of why whiskies taste the way do, you should find this book to be a good general resource.

While I’m on the subject, Dave Broom’s Whisky: The Manual is also a good read. The title is misleading though – as one Amazon reviewer noted, it should be called “Mixed Drinks: The Manual”. Again, though highly personal and subjective, it details the author’s sequential experience with mixing a large number of malts and blends with 5 specific agents: soda water (club soda), coke, ginger ale, green tea, and coconut water. Some of his conclusions are directly at odds with received wisdom, but it is certainly thought provoking if you like to, uhm, mix it up a little. 😉

Ian Buxton’s 101 Whiskies to Try Before You Die is a lot of fun. The author goes to some pains to explain that this is not a “best of” type of book. The selections are certainly not evenly distributed among whisky types, countries or price. Indeed, there are some ridiculously priced rare whiskies in the list, and some (really) common budget blends. I do like the range of international whiskies included. A highly personal look at what one author considers distinctive. Each whisky is well described, with some interesting perspectives. Take it with a grain of your favourite grain whisky. 😉

if you were to buy just one book on whiskies, I would recommend the latest printing of Dr David Wishart’s Whisky Classified. It is surprising to me that this book is not better known among whisky enthusiasts, but I imagine the statistical methodology is not something most people are familiar with. But as I show on this site, the approach taken by Dr Wishart is the most appropriate to clasifying whisky. The book also has plenty of interesting background on whisky making, and detailed discussions of over a hundred individual whiskies (one per distillery).

Lastly, Whisky Advocate magazine is a good read. Tons of articles in every issue, often with a strong travelog feel. As a result, I find they tend to go a bit over the top on the “terroir” aspect (see my discussion here of what to watch out for on this topic). And of course, you get the short blurbs and scores on recently whiskies (although those are also available for free on the website).

Hope you find those to be good starting off points!

1 2