{"id":92,"date":"2015-06-16T22:12:33","date_gmt":"2015-06-16T22:12:33","guid":{"rendered":"http:\/\/whiskyanalysis.com\/wp\/?page_id=92"},"modified":"2015-07-19T00:02:15","modified_gmt":"2015-07-19T04:02:15","slug":"methodology-introduction","status":"publish","type":"page","link":"https:\/\/whiskyanalysis.com\/index.php\/methodology-introduction\/","title":{"rendered":"Methodology &#8211; Introduction"},"content":{"rendered":"<p>Doing the sort of scientifically-valid meta-analysis envisaged here properly is a very labour intensive process. Most online &#8220;metacritic&#8221; sites are simple aggregators that just average a subset of scores on a specific topic (often of variable quality). The goal here is a lot more ambitious \u2013 to weigh the quality of evidence from multiple sources, and provide statistically valid extrapolations. I plan to do this for both quality assessment metrics, as well as underlying flavour profiles. This involves the extensive manual curation of data from numerous sources, which I have been working on for many months.<\/p>\n<p>As mentioned earlier, you might think that it is not possible to provide objective classifications and rankings from purely subjective data. But that too is a question that can be empirically tested using statistical means \u2013 and one that you can easily verify yourself with the right tests. Note that when it comes to a flavour assessment, I am not the first one to think of doing this. I have the advantage of building on the best scientific analysis performed to date on <a href=\"http:\/\/whiskyanalysis.com\/index.php\/methodology-introduction\/methodology-flavour-comparison\/\">whisky flavour<\/a> (which I will describe on these pages). But what is new here is the meta-analysis approach I&#8217;ve taken to quantifying whisky quality. This is something I honestly did not expect to work out as well as it has, and I will need to take some pains to explain that methodology and the results in detail.<\/p>\n<p>Please check out the subheadings of this section for more information on how I have done the meta-analysis here, with some example of the data validations.<\/p>\n<ul>\n<li><a href=\"http:\/\/whiskyanalysis.com\/index.php\/methodology-introduction\/reviewer-selection\/\">Reviewer Selection<\/a> criteria<\/li>\n<li><a href=\"http:\/\/whiskyanalysis.com\/index.php\/methodology-introduction\/methodology-biases-and-limitations\/\">Biases and Limitations<\/a><\/li>\n<li><a href=\"http:\/\/whiskyanalysis.com\/index.php\/methodology-introduction\/methodology-flavour-comparison\/\">Modern Flavour Classification<\/a><\/li>\n<li><a href=\"http:\/\/whiskyanalysis.com\/index.php\/methodology-introduction\/methodology-metacritic-score-construction\/\">Meta-Critic Score<\/a><\/li>\n<\/ul>\n<p>&nbsp;<span hidden class=\"__iawmlf-post-loop-links\" data-iawmlf-links=\"[{&quot;id&quot;:588,&quot;href&quot;:&quot;http:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/methodology-flavour-comparison&quot;,&quot;archived_href&quot;:&quot;https:\\\/\\\/web-wp.archive.org\\\/web\\\/20260524232910\\\/https:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/methodology-flavour-comparison\\\/&quot;,&quot;redirect_href&quot;:&quot;&quot;,&quot;checks&quot;:[{&quot;date&quot;:&quot;2026-05-26 12:25:47&quot;,&quot;http_code&quot;:200}],&quot;broken&quot;:false,&quot;last_checked&quot;:{&quot;date&quot;:&quot;2026-05-26 12:25:47&quot;,&quot;http_code&quot;:200},&quot;process&quot;:&quot;done&quot;},{&quot;id&quot;:1303,&quot;href&quot;:&quot;http:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/reviewer-selection&quot;,&quot;archived_href&quot;:&quot;https:\\\/\\\/web-wp.archive.org\\\/web\\\/20260703002205\\\/https:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/reviewer-selection\\\/&quot;,&quot;redirect_href&quot;:&quot;https:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/reviewer-selection&quot;,&quot;checks&quot;:[{&quot;date&quot;:&quot;2026-07-03 05:07:08&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-07-06 05:40:47&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-07-09 06:16:10&quot;,&quot;http_code&quot;:200}],&quot;broken&quot;:false,&quot;last_checked&quot;:{&quot;date&quot;:&quot;2026-07-09 06:16:10&quot;,&quot;http_code&quot;:200},&quot;process&quot;:&quot;done&quot;},{&quot;id&quot;:1295,&quot;href&quot;:&quot;http:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/methodology-biases-and-limitations&quot;,&quot;archived_href&quot;:&quot;https:\\\/\\\/web-wp.archive.org\\\/web\\\/20260524233416\\\/https:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/methodology-biases-and-limitations\\\/&quot;,&quot;redirect_href&quot;:&quot;&quot;,&quot;checks&quot;:[{&quot;date&quot;:&quot;2026-07-03 00:17:16&quot;,&quot;http_code&quot;:200}],&quot;broken&quot;:false,&quot;last_checked&quot;:{&quot;date&quot;:&quot;2026-07-03 00:17:16&quot;,&quot;http_code&quot;:200},&quot;process&quot;:&quot;done&quot;},{&quot;id&quot;:1296,&quot;href&quot;:&quot;http:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/methodology-metacritic-score-construction&quot;,&quot;archived_href&quot;:&quot;https:\\\/\\\/web-wp.archive.org\\\/web\\\/20260524233418\\\/https:\\\/\\\/whiskyanalysis.com\\\/index.php\\\/methodology-introduction\\\/methodology-metacritic-score-construction\\\/&quot;,&quot;redirect_href&quot;:&quot;&quot;,&quot;checks&quot;:[{&quot;date&quot;:&quot;2026-07-03 00:17:29&quot;,&quot;http_code&quot;:200}],&quot;broken&quot;:false,&quot;last_checked&quot;:{&quot;date&quot;:&quot;2026-07-03 00:17:29&quot;,&quot;http_code&quot;:200},&quot;process&quot;:&quot;done&quot;}]\"><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Doing the sort of scientifically-valid meta-analysis envisaged here properly is a very labour intensive process. Most online &#8220;metacritic&#8221; sites are simple aggregators that just average a subset of scores on a specific topic (often of variable quality). The goal here is a lot more ambitious \u2013 to weigh the quality of evidence from multiple sources, and provide statistically valid extrapolations.<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","meta":{"footnotes":""},"class_list":["post-92","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/pages\/92","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/comments?post=92"}],"version-history":[{"count":11,"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/pages\/92\/revisions"}],"predecessor-version":[{"id":503,"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/pages\/92\/revisions\/503"}],"wp:attachment":[{"href":"https:\/\/whiskyanalysis.com\/index.php\/wp-json\/wp\/v2\/media?parent=92"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}