Forum Archive :
Rollouts
From: 
rambiz 
Address: 
rambiz@gmail.com 
Date: 
3 February 2011 
Subject: 
standard error reported by gnu 
Forum: 
BGonline.org Forums 
What exactly is the standard error reported by gnu while doing rollouts?
Say you rollout out two moves and gnu reports:
move 1: 0.70 winning chance 0.005 SE
move 2: 0.68 winning chance 0.008 SE
Can some one elaborate please? Regardless of the number of games rolled
out, how sure can I be, that move one is better than the other? Please
notice, that 0.68 + 0.008 + 0.005 < 0.7. For the sake of simplicity I've
assumed a cubeless rollout at DMP with no possible gammons.


Tom Keith writes:
Suppose you roll out two plays and want to know whether they are correctly
ranked by their rollout results. (The plays could be wrongly ranked if the
poorer play had luckier dice in the rollout.).
What you can do is compue a "joint standard deviation" (JSD) of the two
plays. If the individual standard deviations are SD1 and SD2,
JSD = sqrt( SD12 + SD22 ).
Then take D, the difference between the rollout results, and divide by the
JSD. Consult the following table to find the probability the plays are
correctly ranked.
Probability the plays
D / JSD are correctly ranked
 
0.0 50%
0.5 69%
1.0 84%
1.5 93.3%
2.0 97.7%
2.5 99.4%
Your example:
If R1 = 0.70 and SD1 = 0.005,
and R2 = 0.68 and SD2 = 0.008, then
JSD = sqrt( 0.0052 + 0.0082 ) = 0.0094
D = 0.70  0.68 = 0.02
D / JSD = 0.02 / 0.0094 = 2.13
From the table, there is roughly a 98% chance that an infinite rollout
uphold the order of these plays.




Rollouts
 Advice (David Montgomery, Apr 1996)
 Cautionary tale (Kit Woolsey, Sept 1995)
 Combining rollouts (Gregg Cattanach+, Dec 2003)
 Confidence intervals (Bob Koca, Nov 2010)
 Confidence intervals (Timothy Chow, May 2010)
 Confidence intervals (Gerry Tesauro, Feb 1994)
 Cubeless vs centeredcube rollouts (Ron Karr, Dec 1997)
 Duplicate dice (David Montgomery, June 1998)
 How reliable are rollouts? (David Montgomery, Aug 1999)
 Level5 versus level6 rollouts (Michael J. Zehr, June 1998)
 Level5 versus level6 rollouts (Chuck Bower, Aug 1997)
 Positions with inaccurate rollouts (Douglas Zare, Oct 2002)
 Reporting results of rollouts (David Montgomery, June 1995)
 Rollout settings (Lokicol+, Apr 2010)
 Settlement limit (Michael J. Zehr, Apr 1998)
 Settlement limit (Kit Woolsey, Dec 1997)
 Settlement limit in races (Alexander Nitschke, Dec 1997)
 Some guidelines (Kit Woolsey, Apr 1996)
 Standard error and JSD (rambiz+, Feb 2011)
 Standard error and JSD (Stick+, Oct 2007)
 Systematic error (Chuck Bower, Oct 1996)
 Tips for doing rollouts (Douglas Zare, June 2002)
 Truncated rollouts (Gregg Cattanach, Oct 2002)
 Truncated rollouts: pros and cons (Jason Lee+, Jan 2006)
 What is a rollout? (Gregg Cattanach, Dec 1999)
From GammOnLine
Long message
Recommended reading
Recent addition

 
