Regression at level 15 . Semeai gets weaker when level increase beyond 10

[alain]

Regression result for r2376 at level 15 (taken in #137) shows that semai is weaker than level 10.

• First possible explanation was "semai-node-limit is too low => random result". It seems not. Semeai.tst run at level 13 with default, or 10 000 or 100 000 semai nodes fails identically, just need more time (1 min , 3 min, 7 min)

[alain]

./regress.pike semeai.tst -o "--mode gtp --level 13 --semeai-node-limit 100000 -M 128"
semeai:28       FAIL 0 0 PASS [1 1 S9]
semeai:31       FAIL 0 0 PASS [1 1 C1]
semeai:32       PASS 1 0 B6 [1 0 B6]
semeai:59       FAIL 0 0 PASS [1 1 H5]
semeai:107      PASS D6 [D6|C7]
semeai:120      FAIL 1 0 A14 [0 0 PASS]
semeai:126      FAIL 1 0 T15 [2 2 (P19|S17|T17)]
semeai:127      FAIL 1 0 S19 [2 2 (O15|O14|R13)]
semeai:133      FAIL 2 2 H2 [1 2 H2]
semeai:134      PASS 1 1 H2 [1 1 H2]
semeai:147      FAIL 0 0 PASS [1 1 (B15|C15|D15|E15)]
semeai                                 431.86  56577975  225706   294820
Total nodes: 56577975 225706 294820
Total time: 431.86 (450.82)
Total uncertainty: 0.00
3 PASS
8 FAIL

[alain]

Semeai.tst results at various level:

 - 10 : 0/0
 - 11 : +1 / -2  :                                              -133 +134 -147
 - 12 : +2 / -4  :     -31 +32                   -120           -133 +134 -147
 - 13 : +3 / -7  :     -31 +32     -59 +107      -120 -126 -127 -133 +134 -147
 - 14 : +3 / -9  :     -31 +32 -34 -59 +107 -124 -120 -126 -127 -133 +134 -147
 - 15 : +3 / -10 : -28 -31 +32 -34 -59 +107 -124 -120 -126 -127 -133 +134 -147

[nando]

What Arend said about branching factor would be correct if the semeai code was actually testing more moves per position. The constant MAX_SEMEAI_MOVES is indeed twice as big as MAX_MOVES (which applies to "simple" OWL reading), resp. 6 and 3, but because of the design in the code, it may happen that the OWL reading is sometimes testing 4, 5 or even 6 different moves, while the semeai code has a very strict policy that prevents it to test more than 3 at any stage (and lower once some depth limits have been reached)

    /* Do not try too many moves. */
    if (tested_moves > 2
	|| (stackp > semeai_branch_depth2 && tested_moves > 1)
	|| (stackp > semeai_branch_depth && tested_moves > 0)) {
      /* If allpats, try and pop to get the move in the sgf record. */
      if (!allpats)
	break;

Time to experiment with different policies ?

[alain]

Increasing various XX_DEPTH and/or trying more moves does not siqgnificantly change the results