Backgammon Rollouts

Rollouts

Truncated rollouts: pros and cons

From:   Jason Lee
Address:   jleemath@gmail.com
Date:   11 January 2006
Subject:   Truncation in rollouts
Forum:   GammOnLine

Let's put our chairs in a circle and talk about our feelings regarding truncation in rollouts. Pros/cons, etc.

Robert-Jan Veldhuizen writes:

Pro: Much faster to reach statistically significant results. Con: Result depends on reliability of final evaluation. By doing two rollouts, one truncated at even (2n plies), one at odd (2n +/- 1 plies) and then averaging the result (the "Tom Keith method"), you can at least reduce certain GNUBG biases in evaluations. The pro is clear, so discussion is mostly about the con. Evaluations are usually better at determining relative equity rather than absolute equity, so an old wisdom says it makes more sense to use truncation for move decisions and not for cube decisions. Some of my own contra-indicators for truncated rollouts are: * lopsided matchscores: the NN is based on cubeless money, so you may want to play it out until the end at non-$ like matchscores * decisions about moves that lead to different types of games: evaluations may be biased between different gametypes. This can happen from the opening move right away: slot vs. split f.i. * positions that are not well understood by evaluations, such as various backgames or prime vs. prime, but also some advanced anchor holding games and bearoffs versus contact, etc. * important future cube decisions ahead (in a cubeful rollout): this can always be a problem, but truncation makes it worse I think as it tends to justify bad cube decisions by the bot Most of all, I think that truncated rollouts using high play settings are almost always a waste of time. Better do a full rollout at 0-ply play then IMO.

Gregg Cattanach writes:

I did a brief truncated to non-trucated rollout test with Snowie a while back. I took 16 of the old GammonLine quiz questions and did each one truncated and non-truncated. In 14 of 16 cases the two rollouts agreed about the top play; the two that didn't had differences of .002 and .004. There was some disagreement about the ranking of the lower ranked plays. I certainly don't use it all the time, but for my 'first pass' rollouts of positions I run at tournaments, I usually do 360 games, 2-ply, precise, truncate at 11. If I see something funny, I might do another one full with more games. Truncation depends on the evaulation at the truncation point, but EVERYTHING the bot does during a rollout depends on its evaulations, anyway, so if you don't trust the truncation point numbers, I guess you can't trust any of the plays it makes, either. If two plays lead to completely different positions, (one a race and one a back-game), I can see where that could cause some problems. I think for checker plays where the relative equity is the only important thing, it's not a bad process, and you have much less noise, of course, in the result. I never truncate cube actions, though, as the absolute equity is the important value. If you want to look at the rollout comparisons, here's a Trunc-vs-Non- Trunc Summary Report: http://pages.prodigy.net/gcattanach/pics/trunc_vs_full.pdf

Chuck Bower writes:

Don't use truncation for cube decisions.

Did you find the information in this article useful?

Do you have any comments you'd like to add?