Forum Archive : Programming

Measuring Difficulty

From:   John Robson
Date:   14 February 2005
Subject:   Measuring Difficulty
Forum:   GammOnLine

Has anyone at the Gnubg team or anyone else in the backgammon community
ever tried to find a way of measuring "difficulty" in a game?

There's no point being rated supernatural if all the moves were no
brainers whereas there's some staifaction being rated "World Class" when
you really did find the best moves amongst may difficult choices.


Peter Grotrian  writes:

Yes, Douglas Zare has in a way in his GammonVillage article,
Square Match Length.

I hope it's ok if I post some snippets from the summary:

The Square Match Length is a measure of the swings in a match. A match
with many swings tends to have a higher SML.

Matches in which one side makes more than 50% mwc more mistakes than the
other tend to have an unusually high Square Match Length. Normalizing
the net skill by the SML decreases the number of matches needed to get a
sensible estimate of your advantage.

When you have a marginal double against a weaker player, you may want to
weigh the decreased Square Match Length if you double (times your
advantage per SML) against the size of your mistake if you don't double.
It is not sufficient to consider whether a position is more complicated
than average. A stronger player should be more averse to doubling any
position when the cube is already high.


Řystein Johansen  writes:

I guess everyone in the GNU Backgammon team have thought about this. It
has also been discussed at the mailing list several times. However we
have not been able to work out a cute way of measuring the difficulty.

A human usually have problems deciding what to do when he can choose
among different game plans, like:

- should I run for the race or should I wait for a shot?
- should I improve my prime or should I try to escape from his prime?

kind of decisions.

Maybe it's possible to look at the different additional nn inputs and
the position classification of the resulting positions to check if the
gameplan is different? Still not easy.

There is currently no plan in the GNU Backgammon project to add such a


Jordan Lampe  writes:

We do, however, have a fairly simple metric for whether GnuBG finds a
position difficult or not: Compare a 0-ply evaluation to a 1-ply
evaluation. If they are far apart, then we know that GnuBG finds the
position difficult. If they are close together then GnuBG usually (but
not always!) finds the position easy.
Did you find the information in this article useful?          

Do you have any comments you'd like to add?     



Adjusting to a weaker opponent  (Brian Sheppard, July 1997) 
Anticomputer positions  (Bill Taylor+, June 1998) 
BKG 9.8 vs. Villa  (Raccoon+, Aug 2006) 
BKG 9.8 vs. Villa  (Andreas Schneider, June 1992) 
BKG beats world champion  (Marty Storer, Sept 1991) 
Backgames  (David Montgomery+, June 1998)  [Long message]
Blockading feature  (Sam Pottle+, Feb 1999)  [Long message]
Board encoding for neural network  (Brian Sheppard, Feb 1997) 
Bot weaknesses  (Douglas Zare, Mar 2003) 
Building and training a neural-net player  (Brian Sheppard, Aug 1998) 
How to count plies?  (Chuck Bower+, Jan 2004)  [GammOnLine forum]
How to count plies?  (tanglebear+, Mar 2003) 
Ideas for improving computer play  (David Montgomery, Feb 1994) 
Ideas on computer players  (Brian Sheppard, Feb 1997) 
Introduction  (Gareth McCaughan, Oct 1994) 
Measuring Difficulty  (John Robson+, Feb 2005)  [GammOnLine forum]
Methods of encoding positions  (Gary Wong, Jan 2001) 
N-ply algorithm  (eXtreme Gammon, Jan 2011) 
Neural net questions  (Brian Sheppard, Mar 1999) 
Pruning the list of moves  (David Montgomery+, Feb 1994) 
Search in Trees with Chance Nodes  (Thomas Hauk, Feb 2004) 
Source code  (Gary Wong, Dec 1999) 
TD-Gammon vs. Robertie  (David Escoffery, June 1992) 
Training for different gammon values  (Gerry Tesauro, Feb 1996) 
Training neural nets  (Walter Trice, Nov 2000) 
Variance reduction in races  (David Montgomery+, Dec 1998)  [Long message]
Variance reduction of rollouts  (Michael J. Zehr+, Aug 1998)  [Long message]
Variance reduction of rollouts  (Jim Williams, June 1997) 
What is a "neural net"?  (Gary Wong, Oct 1998) 
Writing a backgammon program  (Gary Wong, Jan 1999) 

[GammOnLine forum]  From GammOnLine       [Long message]  Long message       [Recommended reading]  Recommended reading       [Recent addition]  Recent addition

  Book Suggestions
Computer Dice
Cube Handling
Cube Handling in Races
Extreme Gammon
Fun and frustration
GNU Backgammon
Luck versus Skill
Magazines & E-zines
Match Archives
Match Equities
Match Play
Match Play at 2-away/2-away
Opening Rolls
Pip Counting
Play Sites
Probability and Statistics
Source Code
Strategy--Bearing Off
Strategy--Checker play


Return to:  Backgammon Galore : Forum Archive Main Page