Forum Archive : Rollouts

Truncated rollouts: pros and cons

From:   Jason Lee
Date:   11 January 2006
Subject:   Truncation in rollouts
Forum:   GammOnLine

Let's put our chairs in a circle and talk about our feelings regarding
truncation in rollouts. Pros/cons, etc.

Robert-Jan Veldhuizen  writes:

Pro: Much faster to reach statistically significant results.

Con: Result depends on reliability of final evaluation.

By doing two rollouts, one truncated at even (2n plies), one at odd (2n
+/- 1 plies) and then averaging the result (the "Tom Keith method"), you
can at least reduce certain GNUBG biases in evaluations.

The pro is clear, so discussion is mostly about the con.

Evaluations are usually better at determining relative equity rather
than absolute equity, so an old wisdom says it makes more sense to use
truncation for move decisions and not for cube decisions.

Some of my own contra-indicators for truncated rollouts are:

  * lopsided matchscores: the NN is based on cubeless money, so you
    may want to play it out until the end at non-$ like matchscores
  * decisions about moves that lead to different types of games:
    evaluations may be biased between different gametypes. This can
    happen from the opening move right away: slot vs. split f.i.
  * positions that are not well understood by evaluations, such as
    various backgames or prime vs. prime, but also some advanced anchor
    holding games and bearoffs versus contact, etc.
  * important future cube decisions ahead (in a cubeful rollout): this
    can always be a problem, but truncation makes it worse I think as it
    tends to justify bad cube decisions by the bot

Most of all, I think that truncated rollouts using high play settings
are almost always a waste of time. Better do a full rollout at 0-ply
play then IMO.

Gregg Cattanach  writes:

I did a brief truncated to non-trucated rollout test with Snowie a while
back. I took 16 of the old GammonLine quiz questions and did each one
truncated and non-truncated.

In 14 of 16 cases the two rollouts agreed about the top play; the two
that didn't had differences of .002 and .004. There was some
disagreement about the ranking of the lower ranked plays.

I certainly don't use it all the time, but for my 'first pass' rollouts
of positions I run at tournaments, I usually do 360 games, 2-ply,
precise, truncate at 11. If I see something funny, I might do another
one full with more games.

Truncation depends on the evaulation at the truncation point, but
EVERYTHING the bot does during a rollout depends on its evaulations,
anyway, so if you don't trust the truncation point numbers, I guess you
can't trust any of the plays it makes, either. If two plays lead to
completely different positions, (one a race and one a back-game), I can
see where that could cause some problems.

I think for checker plays where the relative equity is the only
important thing, it's not a bad process, and you have much less noise,
of course, in the result. I never truncate cube actions, though, as the
absolute equity is the important value.

If you want to look at the rollout comparisons, here's a Trunc-vs-Non-
Trunc Summary Report:

Chuck Bower  writes:

Don't use truncation for cube decisions.
Did you find the information in this article useful?          

Do you have any comments you'd like to add?     



Advice  (David Montgomery, Apr 1996)  [Long message]
Cautionary tale  (Kit Woolsey, Sept 1995) 
Combining rollouts  (Gregg Cattanach+, Dec 2003)  [GammOnLine forum]
Confidence intervals  (Bob Koca, Nov 2010) 
Confidence intervals  (Timothy Chow, May 2010) 
Confidence intervals  (Gerry Tesauro, Feb 1994) 
Cubeless vs centered-cube rollouts  (Ron Karr, Dec 1997) 
Duplicate dice  (David Montgomery, June 1998) 
How reliable are rollouts?  (David Montgomery, Aug 1999) 
Level-5 versus level-6 rollouts  (Michael J. Zehr, June 1998) 
Level-5 versus level-6 rollouts  (Chuck Bower, Aug 1997) 
Positions with inaccurate rollouts  (Douglas Zare, Oct 2002) 
Reporting results of rollouts  (David Montgomery, June 1995) 
Rollout settings  (Lokicol+, Apr 2010) 
Settlement limit  (Michael J. Zehr, Apr 1998) 
Settlement limit  (Kit Woolsey, Dec 1997) 
Settlement limit in races  (Alexander Nitschke, Dec 1997) 
Some guidelines  (Kit Woolsey, Apr 1996) 
Standard error and JSD  (rambiz+, Feb 2011) 
Standard error and JSD  (Stick+, Oct 2007) 
Systematic error  (Chuck Bower, Oct 1996) 
Tips for doing rollouts  (Douglas Zare, June 2002) 
Truncated rollouts  (Gregg Cattanach, Oct 2002) 
Truncated rollouts: pros and cons  (Jason Lee+, Jan 2006)  [GammOnLine forum]
What is a rollout?  (Gregg Cattanach, Dec 1999) 

[GammOnLine forum]  From GammOnLine       [Long message]  Long message       [Recommended reading]  Recommended reading       [Recent addition]  Recent addition

  Book Suggestions
Computer Dice
Cube Handling
Cube Handling in Races
Extreme Gammon
Fun and frustration
GNU Backgammon
Luck versus Skill
Magazines & E-zines
Match Archives
Match Equities
Match Play
Match Play at 2-away/2-away
Opening Rolls
Pip Counting
Play Sites
Probability and Statistics
Source Code
Strategy--Bearing Off
Strategy--Checker play


Return to:  Backgammon Galore : Forum Archive Main Page