Ticket #137 (new task)

Opened 4 years ago

Last modified 18 months ago

Olympiad 2006 9x9 games

Reported by: alain Owned by: gnugo
Priority: normal Milestone: Future
Component: regressions Version:
Severity: normal Keywords:
Cc: patch: yes

Description (last modified by alain) (diff)

Torino 2006 Olympiad games
Played by gnugo 3.7.10 at level 15 (or 16 for some)

Regression at level 15 shows problem with semeai. This topic moved to #165.

Attachments

olympiad2006-9x9.tgz Download (3.6 KB) - added by alain 4 years ago.
9x9 games, with results and correct naming.
olympiad2006.tst Download (7.0 KB) - added by alain 4 years ago.

Regression Results

Attachment Rev. PASS FAIL Nodes Status

Change History

Changed 4 years ago by alain

9x9 games, with results and correct naming.

follow-up: ↓ 4   Changed 4 years ago by alain

At level 10 gnugo 3.7.10 PASS 5/26 tests. During the tournament it was level 15 and 16.

  • test 140 is very weird tenuki during the game at level 16. no pb at level 10.
  • test 260 is difficult kill for me. Need some check by dan(s) player.

OT: Are the 19x19 games available ?

  Changed 4 years ago by alain

  • patch set

  Changed 4 years ago by alain

  • component changed from source to regressions
  • milestone changed from 3.8 to 3.7.11

in reply to: ↑ 1   Changed 4 years ago by gunnar

Replying to alain:

OT: Are the 19x19 games available ?

Yes, they have turned up at  http://www.grappa.univ-lille3.fr/icga/tournament.php?id=3. Now added to CVS, as are the 9x9 games.

follow-up: ↓ 6   Changed 4 years ago by nando

Regression result for r2376 at level 15 

reading:173     FAIL 0 [1 (A2|C7|B1)]
reading:198     FAIL 2 M6 [0]
reading:201     PASS 1 G17 [1 G17]
reading:207     PASS 3 C4 [3 (B4|C4|C1)]
reading:209     PASS 3 B4 [3 (B4|C4)]
reading:217     PASS 2 D4 [2 D4]
reading:219     PASS 1 M14 [1 M14]
reading:232     FAIL 0 [1 (B15|C15|D15|E15)]
reading:233     FAIL 2 B19 [0]
reading                                  8.34    647008       0        0
owl:109         PASS 1 C4 [1 (B5|C4)]
owl:132         PASS 1 R4 [(1|2) R4]
owl:166         PASS 2 P18 [2 P18]
owl                                    380.07  93659610   35960   157478
ld_owl:151      FAIL critical C19 B19 [alive]
ld_owl:190      FAIL critical B3 D2 [critical (B3|D2|B5|A5|A4) A2]
ld_owl:315      FAIL 0 [1 P18]
ld_owl                                 155.15  35377868   41882    12362
optics                                   7.16   1707276       0     3074
filllib:1       FAIL PASS [T15|T13|T10|T8|T5|T1|Q1|N1|A15|A16]
filllib                                 31.44   6044412    2051     8567
atari_atari                            137.61  34441362    2481    31840
connection:52   PASS 1 Q15 [1 Q15]
connection:62   FAIL 1 S15 [1 (T15|T17)]
connection:94   FAIL 0 [1 D9]
connection:105  FAIL 2 G7 [1 G7]
connection:113  FAIL 3 L4 [1 L4]
connection:119  PASS 1 N10 [1 N10]
connection                              67.53  17598574       0    75900
break_in:100    PASS 0 [0]
break_in                                10.16   2815967     814    18593
blunder                                132.16  33204857    6813    21170
unconditional                            2.40     39236       0        0
trevora:200     FAIL F7 [E5]
trevora:370     PASS F6 [F6]
trevora:450     FAIL E8 [G2|G4]
trevora:531     PASS 1 [1]
trevora                                453.71 120540390  143927   225527
nngs1:16        FAIL D5 [M4]
nngs1:20        PASS F17 [F17]
nngs1                                 1246.20 340908686  187063  1161254
strategy:14     FAIL B3 [E9]
strategy:34     FAIL N15 [E17]
strategy                               846.28 224416866  188000   672169
endgame:980     PASS F4 [F4]
endgame:992     FAIL J6 [E5]
endgame                                148.17  35573443    3804   148097
heikki                                  18.49   5253659    2696     4323
neurogo:13      PASS M8 [M7|M8]
neurogo:28      PASS O2 [O2]
neurogo                               1047.32 242821127  244159   832690
arb:231         PASS F5 [F5]
arb                                    173.41  39201704   10046    71320
rosebud                                 37.52  10074068   11552    21036
golife:1        PASS C7 [C7]
golife:2        FAIL A6 [H5]
golife                                  59.08  14298165   27225    20219
arion:1         FAIL D2 [Q6]
arion                                  122.81  32626024   13992    38305
viking                                 220.86  62515443   27171   347961
ego                                    150.78  41607567   12029   258459
dniwog:6        PASS G1 [G1]
dniwog                                 220.66  57359257   55078   121772
lazarus:4       FAIL O11 [R12|Q12|M8]
lazarus:7       PASS T5 [T5]
lazarus:9       FAIL T5 [D15|F15]
lazarus                                769.99 172090912  193439   254529
trevorb:120     FAIL E2 [N2]
trevorb:510     FAIL K7 [C6|B6]
trevorb:590     PASS M9 [H12|M9]
trevorb                                648.06 166829875   88421   702911
strategy2:51    PASS P3 [P3]
strategy2:55    PASS C12 [C12]
strategy2:74    FAIL Q14 [F7|R17|P15]
strategy2:89    FAIL F5 [(D16|B7|G13)]
strategy2:97    FAIL E16 [T8]
strategy2                             1066.97 259671346  247124   845111
nicklas1:901    FAIL D3 [H8]
nicklas1:1302   FAIL G9 [J3]
nicklas1:1405   FAIL B6 [B7]
nicklas1:1509   PASS T14 [T14]
nicklas1:2002   FAIL H6 [J5]
nicklas1                               571.37 133330157  124553   399252
nicklas2:701    PASS F1 [F1]
nicklas2:2401   FAIL G6 [G3|G2]
nicklas2                               158.87  36410579   48799    71246
nicklas3:1403   PASS H9 [J6|H9]
nicklas3                                24.91   5848560   11624      858
nicklas4:805    FAIL D8 [C5|B4]
nicklas4                               122.02  32991287   26563   109375
nicklas5                               228.79  55056167   59865   120799
manyfaces:7     FAIL G16 [R2|S7]
manyfaces                              217.51  61800812   55482   246023
niki                                   346.74  86793401   79563   324491
trevor:9        PASS E5 [E5]
trevor:320      FAIL G1 [J2|H3]
trevor:360      PASS 3 J9 [(2|3) J9]
trevor:730      PASS 3 H1 [(2|3) H1]
trevor:1060     PASS 1 [1]
trevor                                 476.30 117144689  110084   276406
tactics:3       FAIL N17 [M18]
tactics                                154.25  37742284   15731   106947
buzco:1         FAIL N3 [P2]
buzco                                  249.37  66951531   43558   119662
nngs:230        FAIL P4 [E9|G7|H2|K6|N5|D6]
nngs:302        PASS M7 [F15|M7|B14]
nngs:308        PASS L3 [L3|M19]
nngs:330        FAIL D15 [B15|B16]
nngs:510        FAIL P13 [G14]
nngs:590        PASS G3 [G3]
nngs:850        FAIL J18 [H15]
nngs:900        FAIL S14 [Q15]
nngs:1060       PASS F3 [H3|F3]
nngs:1700       FAIL J18 [D8|E9|C9]
nngs:1880       PASS F6 [F6|G7]
nngs                                  3202.41 848239303  543885  2185576
trevorc:110     FAIL B7 [A10]
trevorc:500     PASS J2 [J2]
trevorc:880     FAIL G1 [J10]
trevorc:910     FAIL E1 [H13]
trevorc:940     PASS G1 [G1]
trevorc:1250    FAIL C2 [!C2]
trevorc:1440    PASS E12 [E12|F11]
trevorc:1540    FAIL L10 [M10]
trevorc:1650    PASS F7 [F7]
trevorc:1660    PASS L10 [L10]
trevorc                               1360.29 345700897  265010  1447820
strategy3:119   FAIL J6 [D9|J3]
strategy3:128   FAIL T11 [O8]
strategy3                              743.54 168771931  167239   831697
capture:18      FAIL 0 [1 F6]
capture                                  1.38    234614       0        0
connect:78      PASS 0 [0]
connect                                 11.40   2458669       0    15097
global:1        FAIL A4 [B3]
global:9        PASS F4 [F4]
global:44       PASS D5 [D5]
global                                1104.97 268193194  253766   701018
vie:4           PASS 1 G2 [1 G2]
vie:8           FAIL 1 R7 [1 (R8|S9)]
vie                                    117.33  27165953   15314    75806
arend:3         FAIL L5 [P6|Q5]
arend:9         PASS S17 [S17]
arend                                  914.53 238873385   86261  1206285
13x13:1         PASS M2 [M2]
13x13:40        PASS K13 [K13]
13x13:60        FAIL H4 [K5|L5]
13x13:78        PASS L5 [L5]
13x13:87        FAIL A9 [E6]
13x13                                 1280.38 324934029  349664  1220477
semeai:28       FAIL 0 0 PASS [1 1 S9]
semeai:31       FAIL 1 1 J4 [1 1 C1]
semeai:32       PASS 1 0 B6 [1 0 B6]
semeai:34       FAIL 1 0 S18 [1 1 S18]
semeai:59       FAIL 0 0 PASS [1 1 H5]
semeai:107      PASS D6 [D6|C7]
semeai:120      FAIL 1 0 A14 [0 0 PASS]
semeai:124      FAIL 1 0 S17 [1 1 P19]
semeai:126      FAIL 1 0 T15 [2 2 (P19|S17|T17)]
semeai:127      FAIL 1 0 S19 [2 2 (O15|O14|R13)]
semeai:133      FAIL 2 2 H2 [1 2 H2]
semeai:134      PASS 1 1 H2 [1 1 H2]
semeai:147      FAIL 0 0 PASS [1 1 (B15|C15|D15|E15)]
semeai                                 244.92  50252409   66609   103067
trevord                               1264.95 352544183  115345  1949450
strategy4:151   FAIL S1 [T5]
strategy4:153   PASS Q9 [Q9]
strategy4:155   FAIL P2 [D18]
strategy4:156   FAIL G9 [B14|B16]
strategy4:160   FAIL M12 [K2|L2|L3]
strategy4:163   PASS P8 [O7|P8]
strategy4:166   PASS D11 [D11]
strategy4:167   FAIL E5 [D4]
strategy4:179   PASS E2 [P3|O2|P2|O4|E2]
strategy4:182   PASS F8 [E7|F8]
strategy4:190   FAIL E7 [D13]
strategy4:194   PASS A13 [A13]
strategy4:197   FAIL G14 [K3|S18]
strategy4:198   FAIL G14 [C10|S18]
strategy4:199   FAIL G14 [N5|S18]
strategy4                             1699.37 434110531  336377  1542335
owl1:266        FAIL 1 C18 [1 B17]
owl1:293        PASS 0 [0]
owl1:302        PASS 0 [0]
owl1:316        PASS 0 [0]
owl1:318        PASS 0 [0]
owl1:344        FAIL 1 M2 [0]
owl1:345        PASS 1 N13 [1 N13]
owl1:348        PASS 0 [0]
owl1:352        PASS 0 [0]
owl1:358        PASS 0 [0]
owl1:359        PASS 0 [0]
owl1:367        PASS 1 O4 [1 O4]
owl1:379        PASS 0 [0]
owl1                                   252.12  63818256   18881   215285
handtalk:7      FAIL P4 [R4]
handtalk:10     FAIL L5 [E9|F8]
handtalk:15     FAIL M19 [J3]
handtalk                               484.84 126370258  106995   492878
nngs2:150       FAIL E9 [M3|L3]
nngs2:240       PASS F17 [F17]
nngs2:460       FAIL N15 [J4|K2|K3|K4|K5|L2]
nngs2:600       PASS B5 [B5]
nngs2                                 1695.68 427896387  278233   894880
nngs3:160       PASS L15 [L15]
nngs3:170       PASS L15 [L15]
nngs3:230       FAIL L12 [!L12]
nngs3:260       FAIL K11 [G11]
nngs3:340       PASS P10 [P10]
nngs3:350       FAIL P10 [E12]
nngs3:520       PASS T8 [!N3]
nngs3:710       PASS Q2 [Q2]
nngs3:950       PASS F15 [F15]
nngs3:1010      FAIL J15 [H16|H17]
nngs3:1140      PASS B19 [B19]
nngs3:1190      FAIL R19 [D17]
nngs3                                 3832.69 916817766  933431  2436117
nngs4:170       PASS S6 [S6]
nngs4:200       FAIL T2 [T6]
nngs4:230       PASS Q18 [Q18]
nngs4:260       FAIL D8 [A4]
nngs4:410       FAIL PASS [E1]
nngs4:420       FAIL F4 [G6]
nngs4:560       PASS B13 [B13]
nngs4:600       FAIL N3 [L9|L12|L10]
nngs4:680       PASS P9 [P9]
nngs4:690       PASS N6 [N6]
nngs4:790       PASS E15 [E15]
nngs4:800       FAIL C3 [E4]
nngs4:1050      PASS O9 [O9|O10]
nngs4:1060      PASS N9 [N9]
nngs4:1120      PASS N7 [N7]
nngs4                                 3678.47 898893471  906245  2240931
strategy5:225   FAIL L12 [O12]
strategy5:229   FAIL N10 [D7]
strategy5:236   PASS R12 [R12]
strategy5:289   FAIL F4 [M16]
strategy5:292   FAIL S4 [F16|F17]
strategy5                              804.39 214829632  190032   569456
century2002:120 FAIL K2 [L3]
century2002:190 PASS D3 [D3]
century2002                            556.31 145750793  135394   339485
auto01:4        FAIL 1 T2 [1 (R1|N3|N5|M5|N6|Q2)]
auto01:13       FAIL 0 [1 (Q12|Q13)]
auto01                                  21.25   5729947    1654    26468
auto02                                  44.04  12051488    5849    94996
auto03                                  27.01   8552380    1022    29753
auto04:2        PASS 1 G18 [!0]
auto04                                  13.10   3735572     920    11307
auto_handtalk                           49.09  10286429   11526    90731
safety                                  35.11  10217395   10131    50540
ninestones:150  FAIL G14 [C5|D5]
ninestones:320  FAIL D6 [D5]
ninestones:550  FAIL L9 [G5|J4]
ninestones:560  FAIL L9 [A9|B10|D11|F11|J15|A16]
ninestones:570  FAIL H1 [A9|B10|D11|F11|J15|A16]
ninestones:700  PASS K15 [K15]
ninestones:770  PASS Q19 [Q19]
ninestones:790  FAIL K5 [B1]
ninestones:800  PASS K1 [K1]
ninestones                            2660.16 616799038  584255  1256817
tactics1:108    FAIL 0 [1]
tactics1                                81.54  19250177   16358   124732
manyfaces1:33   FAIL Q3 [S17|R3]
manyfaces1:36   FAIL S16 [P16]
manyfaces1:50   PASS 1 N18 [1 .*]
manyfaces1                             379.79  95109646  112970   308898
gunnar:30       FAIL PASS [F18]
gunnar:36       FAIL A14 [B18]
gunnar:45       PASS K1 [K1]
gunnar:51       FAIL T5 [S7]
gunnar:66       FAIL B9 [B11]
gunnar:67       PASS D9 [D9]
gunnar:70       FAIL B10 [A18|A13]
gunnar:71       FAIL F5 [D5]
gunnar:84       PASS P13 [P13]
gunnar:97       FAIL B8 [N16]
gunnar:98       FAIL F7 [A6]
gunnar:99       FAIL T2 [C11]
gunnar:100      FAIL C11 [S6]
gunnar:101      FAIL T7 [T8]
gunnar                                1995.20 507375053  220639  1612198
arend2:30       FAIL O7 [K3|J4|M5]
arend2:220      PASS K11 [K11|L11]
arend2:260      PASS E14 [E14]
arend2                                 595.69 156784334  152936   527729
nando:2         PASS 0 [0]
nando:23        PASS 0 [0]
nando:35        PASS 3 S8 [3 S8]
nando:150       PASS 1 P4 [1 P4]
nando:151       PASS 0 [0]
nando                                  223.51  57252944   32094   334459
thrash:6        FAIL Q13 [T4|M5|O4|O2|P1]
thrash:19       FAIL F7 [J9|F9]
thrash:23       PASS D7 [A5|D7]
thrash:27       FAIL G3 [J6]
thrash:28       FAIL G3 [A2]
thrash:29       FAIL G3 [D3]
thrash                                 319.75  69036731   57188   236314
13x13b:10       FAIL M6 [M4|L6]
13x13b:29       PASS F3 [F3]
13x13b:32       FAIL G5 [H4]
13x13b:35       PASS E8 [D7|E8|F9]
13x13b                                 676.81 167287188  161924   533338
joseki                                 409.69 110450473   28272    56223
gifu03:103      FAIL F18 [F17]
gifu03:304      PASS J7 [J7]
gifu03                                 832.07 222604526  133976   977823
seki:103        PASS A2 [B1|A2]
seki:903        PASS C2 [C2|C1]
seki:1002       PASS B2 [B2]
seki                                    10.69   1175757    6197     3494
9x9:30          PASS D7 [D7]
9x9:130         PASS B9 [B9]
9x9:197         FAIL D6 [E8|H5]
9x9:200         PASS D4 [D4]
9x9:250         PASS A2 [A2|C1|C2]
9x9:260         FAIL D8 [E6]
9x9:320         PASS E5 [E5|F4|D4|E4]
9x9:430         PASS G7 [G7|H7]
9x9:450         PASS B5 [B5|B4]
9x9:580         PASS E6 [E6]
9x9:590         PASS E5 [E5]
9x9:660         FAIL F2 [D3]
9x9                                   2066.70 523889618  565466   529099
cgf2004:70      PASS N4 [N4]
cgf2004                                262.26  70045758   19168   418885
kgs:80          PASS L1 [L1|O2|Q1]
kgs:110         FAIL O8 [G13]
kgs:190         PASS C4 [C4]
kgs:220         FAIL N12 [T10]
kgs:230         PASS J19 [J12|J11|L10|L9|J19]
kgs:290         FAIL D5 [B2]
kgs:330         PASS H1 [H1]
kgs:340         PASS L4 [L4]
kgs:430         FAIL S18 [K18]
kgs:590         PASS P16 [P16]
kgs:610         FAIL D6 [O15|N16]
kgs:690         PASS R11 [R11]
kgs:720         PASS A15 [A15]
kgs:730         PASS R16 [R16]
kgs                                   2600.28 660242249  497437  2117130
olympiad2004:301 PASS D4 [D4]
olympiad2004:601 FAIL F6 [F7|H4|G3]
olympiad2004                           998.42 251314781  184714   644291
tiny                                    13.74   1639510   15436      121
gifu05:1090     PASS M9 [M9]
gifu05:1200     FAIL B18 [C16]
gifu05:1210     FAIL O6 [E17]
gifu05:1230     FAIL M8 [M6]
gifu05                                 342.13  83676813   46654   169627
13x13c:1        FAIL A1 [E12|K4]
13x13c:2        PASS K3 [K3]
13x13c:43       FAIL B5 [G13]
13x13c                                4403.58 909490498   57311    73422
Total nodes: 13115248135 9814327 36525781
Total time: 53024.04 (53195.87)
Total uncertainty: 47.20
139 PASS
143 FAIL

At this level, it feels like:

  • the engine globally is neither weaker nor stronger
  • the OWL reading seems improved (see owl1.tst)
  • the semeai reading seems to suffer (see semeai.tst)

in reply to: ↑ 5   Changed 4 years ago by arend

  • description modified (diff)

At this level, it feels like: * the engine globally is neither weaker nor stronger * the OWL reading seems improved (see owl1.tst) * the semeai reading seems to suffer (see semeai.tst)

  Changed 4 years ago by arend

Grr I accidentally changed the description instead of adding a comment, can this be undone?

  Changed 4 years ago by alain

  • description modified (diff)

  Changed 4 years ago by alain

The semeai problem is probably the following: We increase depth levels and node limits for the semeai module above level 10. However, when the (branching factor)(depth level) grows faster than the node limit, then we will run out of nodes on many more reading problems.

When the semeai reading runs out of nodes, it returns pretty random results.

Arend

  Changed 4 years ago by alain

  • description modified (diff)

Changed 4 years ago by alain

  Changed 4 years ago by alain

Test file Olympiad2006.tst from 9x9 and 19x19 played at level 15 or 16.
Current 2377 PASS 10 of these tests (amongst 51).

WIP: see if/how semeai is implied...

  Changed 4 years ago by alain

Results at level 10 

olympiad2006:50 PASS F7 [F7|G6]
olympiad2006:60 PASS C5 [C5|C3]
olympiad2006:100 PASS E4 [!G3]
olympiad2006:140 PASS D4 [D4]
olympiad2006:170 PASS C6 [!E5]
olympiad2006:230 PASS D7 [D7]
olympiad2006:290 PASS D4 [D4]
olympiad2006:320 PASS F19 [F19]
olympiad2006:460 PASS O14 [O14]
olympiad2006:470 PASS J14 [!S10]
olympiad2006                           153.13  37267159   80395   248971
Total nodes: 37267159 80395 248971
Total time: 153.13 (153.40)
Total uncertainty: 3.92
10 PASS

  Changed 3 years ago by gunnar

  • milestone changed from 3.7.11 to 3.8

  Changed 18 months ago by gunnar

  • milestone changed from 3.8 to Future
Note: See TracTickets for help on using tickets.