Support page for GDC16 “TLDR statistics”

Thanks to everyone who came to my GDC talk.

Here are my PowerPoint slides, with a summary of my narration in the speaker’s notes. You can also download a .PDF export.

Macros

Here’s the further info and how-to on the Excel macros I demonstrated.

The Student’s t-test is built into Microsoft Excel. You can use it in two ways:

The Analysis ToolPak provides a convenient wizard that builds up the nice tables shown in the talk, but is otherwise equivalent to using the T.TEST() function.

Many of the macros depicted in my talk come from the excellent Real Statistics In Excel macro package. You can download it directly from its creator here.

In particular, this page describes the use of the Mann-Whitney U test I depict in the talk.

My custom Excel add-in for calculating confidence intervals on relative risk and difference of proportions tests is here. It provides these functions:

  • RRISK: calculates the relative risk as shown in the talk. It takes four parameters, each of which should be a single cell: ( # of individuals in control group with a NEGATIVE result, # of individuals in control group with a POSITIVE result, # of individuals in experimental group with a NEGATIVE result, # of individuals in experimental group with a POSITIVE result). Note that these are absolute counts and not percentages.
  • RRISK_CONF_LO: calculates the lower bound of the confidence interval around the result of RRISK(). It takes the same four parameters as RRISK(), plus a fifth alpha parameter which is basically the % chance you’re willing to tolerate that the actual ratio lies outside your interval. ie, for a 95% confidence interval, specify 0.05.
  • RRISK_CONF_HI: As RRISK_CONF_LO, but gives you the upper bound of the confidence interval.
  • DIFFPROP_CONF_LO: Same parameters as RRISK_CONF_LO, but gives you the lower bound of the difference of proportions test – ie, use this when you are calculating an absolute difference in percentage points, and not a relative “monocled players are 2.4x as likely to own pants.”
  • DIFFPROP_CONF_HI: As _LO, but gives you the upper bound.
  • T_DIFFMEANS: Gives you the confidence interval of the difference of means in a two-sample t-test – the 8.13 value from the HDD/SSD build times example in my talk. It takes three parameters:
    1. The range of cells containing all the values from sample group A
    2. The range of cells containing all the values from sample group B
    3. alpha, which is basically the % chance you’re willing to tolerate that the actual difference of means lies outside your interval. ie, for a 95% confidence interval, specify 0.05.

Here’s a macro-enabled worksheet demonstrating their use.

Further reading

Real Statistics In Excel is generally an excellent resource for the background of a variety of tests and how to implement them in Excel; I relied on it heavily in my talk preparation.

statisticslectures.com has a nice set of YouTube videos that teach basic statistics in easily digestable chunks.

OnlineStatBook and StatTrek offer fairly legible introductions to all the hypothesis tests I described. In particular I got the math for my confidence macros from OnlineStatBook.

4 Comments

  1. […] 슬라이드 공개했으니 편히 들으라고 하더니 아주 빠르게 달려가는 발표였습니다. 거창한 통계에 대해 얘기하는 게 아니라, 엑셀에서 간단하게 써볼 수 있는 기초 통계에 대한 얘기였어요. t 테스트라든가 귀무가설이라든가 대립가설이라든가 하는 것들 말이죠. 통계 너무 오랜만이라서 가물가물했는데, 슬라이드 올라오면 좀 더 찾아보려고요. […]

  2. […] “TLDR Statistics for Game Devs. The math you need to win arguments” – Elan Ruskin (Insomniac Games) […]

  3. […] TL;DR Statistics for Game Devs: the math you need to win arguments […]

  4. Kent Morita says:

    Thank you so much for this presentation and providing a space to host the files.

    Best,

    Kent

Leave a Reply