Forum Archive :
Has anyone at the Gnubg team or anyone else in the backgammon community
ever tried to find a way of measuring "difficulty" in a game?
There's no point being rated supernatural if all the moves were no
brainers whereas there's some staifaction being rated "World Class" when
you really did find the best moves amongst may difficult choices.
Peter Grotrian writes:
Yes, Douglas Zare has in a way in his GammonVillage article,
Square Match Length.
I hope it's ok if I post some snippets from the summary:
The Square Match Length is a measure of the swings in a match. A match
with many swings tends to have a higher SML.
Matches in which one side makes more than 50% mwc more mistakes than the
other tend to have an unusually high Square Match Length. Normalizing
the net skill by the SML decreases the number of matches needed to get a
sensible estimate of your advantage.
When you have a marginal double against a weaker player, you may want to
weigh the decreased Square Match Length if you double (times your
advantage per SML) against the size of your mistake if you don't double.
It is not sufficient to consider whether a position is more complicated
than average. A stronger player should be more averse to doubling any
position when the cube is already high.
Řystein Johansen writes:
I guess everyone in the GNU Backgammon team have thought about this. It
has also been discussed at the mailing list several times. However we
have not been able to work out a cute way of measuring the difficulty.
A human usually have problems deciding what to do when he can choose
among different game plans, like:
- should I run for the race or should I wait for a shot?
- should I improve my prime or should I try to escape from his prime?
kind of decisions.
Maybe it's possible to look at the different additional nn inputs and
the position classification of the resulting positions to check if the
gameplan is different? Still not easy.
There is currently no plan in the GNU Backgammon project to add such a
Jordan Lampe writes:
We do, however, have a fairly simple metric for whether GnuBG finds a
position difficult or not: Compare a 0-ply evaluation to a 1-ply
evaluation. If they are far apart, then we know that GnuBG finds the
position difficult. If they are close together then GnuBG usually (but
not always!) finds the position easy.
- Adjusting to a weaker opponent (Brian Sheppard, July 1997)
- Anticomputer positions (Bill Taylor+, June 1998)
- BKG 9.8 vs. Villa (Raccoon+, Aug 2006)
- BKG 9.8 vs. Villa (Andreas Schneider, June 1992)
- BKG beats world champion (Marty Storer, Sept 1991)
- Backgames (David Montgomery+, June 1998)
- Blockading feature (Sam Pottle+, Feb 1999)
- Board encoding for neural network (Brian Sheppard, Feb 1997)
- Bot weaknesses (Douglas Zare, Mar 2003)
- Building and training a neural-net player (Brian Sheppard, Aug 1998)
- How to count plies? (Chuck Bower+, Jan 2004)
- How to count plies? (tanglebear+, Mar 2003)
- Ideas for improving computer play (David Montgomery, Feb 1994)
- Ideas on computer players (Brian Sheppard, Feb 1997)
- Introduction (Gareth McCaughan, Oct 1994)
- Measuring Difficulty (John Robson+, Feb 2005)
- Methods of encoding positions (Gary Wong, Jan 2001)
- N-ply algorithm (eXtreme Gammon, Jan 2011)
- Neural net questions (Brian Sheppard, Mar 1999)
- Pruning the list of moves (David Montgomery+, Feb 1994)
- Search in Trees with Chance Nodes (Thomas Hauk, Feb 2004)
- Source code (Gary Wong, Dec 1999)
- TD-Gammon vs. Robertie (David Escoffery, June 1992)
- Training for different gammon values (Gerry Tesauro, Feb 1996)
- Training neural nets (Walter Trice, Nov 2000)
- Variance reduction in races (David Montgomery+, Dec 1998)
- Variance reduction of rollouts (Michael J. Zehr+, Aug 1998)
- Variance reduction of rollouts (Jim Williams, June 1997)
- What is a "neural net"? (Gary Wong, Oct 1998)
- Writing a backgammon program (Gary Wong, Jan 1999)