Forum Archive :
Rollouts
The confidence interval is a range. For example if you see "the 95%
confidence interval is .72 + .03" the confidence interval is [.69, .75].
Requiring that the 95% CI levels do not overlap is a very stringent
condition.
Roughly, to find the margin of error for the difference you should add the
two + values and divide by SQRT(2), approx 1.41. (This works if the +
values are about equal, which will usually be the case if the plays were
rolled out the same number of times.)
As an example:
Play A 95% CI is .70 + .10
Play B 95% CI is .55 + .10
The 95% CI for the difference is .15 + .141 = [.009, .291] This does not
include 0 so you have greater than 95% confidence (barely) that play A is
better.
The + values have a term "margin of error" and for a 95% confidence
interval are calculated as approximately 1.96 * std error. The 1.96 value
comes from the normal distribution and is chosen so that P( 1.96 < Z <
1.96) = .95 where Z is a normally distributed variable with mean 0 and std.
dev 1.
Working backwards here we can determine that the std errors for A and B are
.10/1.96 = .05102
The std error for the difference is (.05102+.05102)/SQRT(2) = .07215 The
difference of .15 is that value times 2.08. Using a normal table (or
software) P(2.08 < Z < 2.08) = .9624 so there is 96.24% confidence in my
example that play A is better than play B.
It is left as an exercise what would be the confidence if the plays were
actually .2 (the sum of the 95% confidence interval margins of error)
apart.




Rollouts
 Advice (David Montgomery, Apr 1996)
 Cautionary tale (Kit Woolsey, Sept 1995)
 Combining rollouts (Gregg Cattanach+, Dec 2003)
 Confidence intervals (Bob Koca, Nov 2010)
 Confidence intervals (Timothy Chow, May 2010)
 Confidence intervals (Gerry Tesauro, Feb 1994)
 Cubeless vs centeredcube rollouts (Ron Karr, Dec 1997)
 Duplicate dice (David Montgomery, June 1998)
 How reliable are rollouts? (David Montgomery, Aug 1999)
 Level5 versus level6 rollouts (Michael J. Zehr, June 1998)
 Level5 versus level6 rollouts (Chuck Bower, Aug 1997)
 Positions with inaccurate rollouts (Douglas Zare, Oct 2002)
 Reporting results of rollouts (David Montgomery, June 1995)
 Rollout settings (Lokicol+, Apr 2010)
 Settlement limit (Michael J. Zehr, Apr 1998)
 Settlement limit (Kit Woolsey, Dec 1997)
 Settlement limit in races (Alexander Nitschke, Dec 1997)
 Some guidelines (Kit Woolsey, Apr 1996)
 Standard error and JSD (rambiz+, Feb 2011)
 Standard error and JSD (Stick+, Oct 2007)
 Systematic error (Chuck Bower, Oct 1996)
 Tips for doing rollouts (Douglas Zare, June 2002)
 Truncated rollouts (Gregg Cattanach, Oct 2002)
 Truncated rollouts: pros and cons (Jason Lee+, Jan 2006)
 What is a rollout? (Gregg Cattanach, Dec 1999)
From GammOnLine
Long message
Recommended reading
Recent addition

 
