One day I was minding my own business kicking a couple of FIBSbots
around, and a human kibitzer asked me why my experience points on the
server hadn't been going up much lately. I told him I had taken some
time off to study the matches I had recorded electronically from FIBS.
(I also told him he was very observant!) He didn't seem convinced that
it took that much time to run a few matches by JellyFish, then rush
back to playing again before the dreaded withdrawal symptoms began in
Analyzing your own matches can indeed be a quick process if you
just click through them and let your bot do all your thinking for you.
But why settle for that, when you are so much smarter? Here is the
method I recommend for letting your bot assist you in your ceaseless
quest for knowledge, enlightenment, and the ability to squash your
opponents like so many little ants:
- Record the match as you play it (electronically is fine). Feed it
through JellyFish, Snowie, or any other bot you may have that lets you
run matches through it conveniently. I will base the rest of this
discussion on JellyFish 3.0, since that is the bot I use. From what I
have seen of them, I'd expect the procedure for Snowie or any other
advanced bot to be pretty similar.
- Help your bot find your errors. Set its gotcha point at no
more than .03, since on its first pass it may not otherwise catch as
much. (I use .028.) If it flags something, verify twice to see if it
"reverses on appeal". If it ends up preferring your play after all when
it gets to Level 7, you can generally assume your play was OK and
continue. But just to be sure, you may want to backtrack and get the
entire "Top Ten" list for the decision, since there may have been a
third choice it likes better than either play! Also note the trend-was
the difference between the top two choices (or the bot's top pick and
yours) rising or falling from Level 5 through Level 7? If the
difference was rising as the bot looked further ahead, chances are your
play was dead wrong. If it was declining sharply, your play may have
actually been correct if the final difference was not large.
- Make the bot tell you about the errors it didn't catch. As
you go through the match, even if the bot doesn't flag a play, if you
are unsure, get the Top Ten list and verify twice to see what Jelly
thinks when it gets to use its maximum lookahead. If the bot's Top Ten
list does not include a play you think it should have considered, make
it do so by feeding it the position after that play, then telling it to
evaluate from the opponent's point of view.
You can instead have it evaluate more than 10 plays each time, but I
choose 10 because then if I want to print the position, it all fits on
one page. Limiting yourself to 10 plays also makes sense if you have a slow
machine, because otherwise it could take quite a while for Jellyfish to
verify from Level 6 to Level 7, and almost all the extra time would be
spent evaluating nonsense plays.
- Toss out the evaluation if you know from experience that
your particular bot is clueless in the type of game in question.
Backgames are still a big weakness in bot play (and in quite a lot of
human play, too). There are also some situations where the bots will
run from an anchor prematurely, though a rollout will usually set them
straight in those cases.
- As best you can, distinguish among careless errors, matters
of style, minor nuance plays, and true concept errors. This last group
is the one you need to worry most about! Concept errors tend to involve
game plan selection, a fork in the road where you make a choice from
which there will be no turning back. If the error is large (I consider
>.70 equity large), I find it helpful to print out the Level 7
evaluation page, then review it 3-6 months later along with all the
others I have been collecting.
- Run evaluations for the same play at double match point (assuming
the play occurred at some other score in the actual match) and also as
a money problem. These cases help you to see how gammons or the lack
thereof affect the decision in question.
For some match scores, especially ones involving 2-, 3-, or 4-away
I find it helpful to do the match equity calculations by hand. This
double for cube decisions, so to speak.
- If you feel it is appropriate and convenient, do both Level
5 cubeful and Level 6 cubeless rollouts of the top contending plays
(and also whatever you did, if it didn't make the list and you still
are not convinced it is with good reason) You definitely need to roll
out long-horizon endgame problems, and any problem where the evaluated
difference was relatively small. I have found that "reversals on
appeal" are quite common at the .04-type level. In all cases, "spot
check" a couple of key plays to be sure the bot is not making massive
errors in the Level 5 or Level 6 rollouts. This can affect the results
big time, especially if one side is much more difficult to play than
the other. Also, be aware of known "glitches" in the software you use,
and check to see if opportunities to make these signature errors are
likely to arise in your rollouts. If so, it may be better to believe
the Level 7 evaluation or just your own judgment rather than the
- If there is still a
significant difference between the best play and the second choice
after the rollouts, determine what features of the position are most
important. Do this by changing one feature at a time and seeing what
effect it has on the play selections.
- Propose a general principle governing the type of play you have
just studied. If possible, come up with one or more radically different
positions where you think the decision should be similar, based on your
new principle, because the key features are similar to those of the
position you studied. Ask the bot if it agrees.
Isn't this all kind of time-consuming? Well, uh, yeah, it is.
Now that FIBS player who thought my experience points weren't going up
very fast knows why. But for me, part of the "experience" gained from
playing is learning as much as possible from my games. Backgammon is so
much more satisfying, at least to me, when instead of getting the mere
intermittent reinforcement that the dice often provide, I also have a
sense of real progress.