Ticket #41: STS2GNUGo.txt

Some explanation about the criteria
I adopted to make the porting of STS-RV.

1) The first step for conversion 
is done with this awk script:

<awk-script>
#!/bin/awk -f
# skip comment lines
/^#[^?]/ { next }
/^# *$/ { next }
{ 
# replace path and function
gsub(/Sgf_test_files/,"games/STS-RV")
gsub(/solve_semeaiS/,"analyze_semeai")
# translate exit code of solve_semeaiS
gsub(/\[0 /,"[0 0 ")
gsub(/\[1 /,"[1 1 ")
gsub(/\[2 /,"[1 0 ")
gsub(/\[3 /,"[3 @ ")
gsub(/\[4 /,"[4 @ ")
# remove espected failures mark
gsub(/\]\*/,"]")
print 
}
</awk-script>
 
2) then I edited manually the special cases
marked with a '@' becouse the Ko (4) could 
be good [2 x ...] or bad [3 x ...].
The Unknow result (3) of solve_semeaiS never
occurred as espected reply in original tests.

3) When the analysis response was correct 
but the move was not in the espected set, 
I checked if the move was anyway a good reply.
This becouse often the set was too small and the
move could be not wrong but simply not the
best, or simply missed in the original test
although good and valid. 

This revision included the very frequent case
that test espected a PASS reply and GNU Go played
some move elsewere, not relevant for the semeai, 
but not wrong. The workaround suggested by Gunnar
is a regexp test like [x y (.*)].

In other cases I simply appended the valid move(s)
to the espected set of good replies.

NOTE: I assumed implicitly that if test is passed
then test is well defined. This is not true in
general becouse is possible that changing the
code of GNU Go, in the future it will play
a correct (different) move not included in the 
set of good replies (like often detected and 
revised as described above).

But to avoid this, *each* test should be revised 
to verify that *all* good moves are included. 
And this require too much work, so it's better
to take benefit from the STS-RV test suite
anyway, despite this small accuracy lack.

-----------

Here the complete list of change made to
the original test suite:


# File: STS-RV_1.tst

4 FAILED: Correct '1 1 (A10|A12|B11)', got '1 1 B8' ==> 1 1 (A10|A12|B11|A8|B8) 
6 FAILED: Correct '1 1 (J2|K1|H1)', got '1 1 M1' ==> 1 1 (J2|K1|H1|M2|M1) 
8 FAILED: Correct '1 1 (S8|T9|T7)', got '1 1 T11' ==> 1 1 (S8|T9|T7|R11|S11|T11)' 
17 FAILED: Correct '1 1 (D14|C14|C13|C12|C11|B10)', got '1 1 B19' ==> 1 1 (D14|C14|C13|C12|C11|B10|B19) # B19 irrelevant but not wrong
61 FAILED: Correct '1 1 T3', got '1 1 O2' ==> 1 1 (T3|O1|O2|P1|P3|R1) 
113 FAILED: Correct '1 1 B2', got '1 1 C3' ==> 1 1 (B2|C3|D2) 
129 FAILED: Correct '1 1 (S18|T16)', got '1 1 T14' ==> 1 1 (S18|T16|T14) 
135 FAILED: Correct '1 1 C2', got '1 1 D2' ==> 1 1 (C2|D2) 
137 FAILED: Correct '1 1 R3', got '1 1 O1' ==> 1 1 (R3|O1) 
163 FAILED: Correct '1 1 C18', got '1 1 C12' ==> 1 1 (C18|D17) 
170 FAILED: Correct '1 0 K18', got '1 0 L17' ==> 1 0 (K18|L17) 
186 FAILED: Correct '1 1 J2', got '1 1 C4' ==> 1 1 (J2|B2|B3|D3)


# File: STS-RV_GSAT.tst

2 FAILED: Correct '1 1 (B1|A2)', got '1 1 C1' ==> 1 1 (B1|A2|C1)
60 FAILED: Correct '1 1 (P2|P3|Q1|R1|S2|S3)', got '1 1 T2' ==> 1 1 (P2|P3|Q1|R1|S2|S3|T2)
68 FAILED: Correct '1 1 (P1|P2|O3|S1|S2)', got '1 1 T3' ==> 1 1 (P1|P2|O3|S1|T2|T3)


# File: STS-RV_e.tst

127 FAILED: Correct '1 0 B4', got '1 0 C6' => 1 0 (B4|C6)
128 FAILED: Correct '1 1 B4', got '1 1 E3' => 1 1 (B4|E3)
168 FAILED: Correct '1 1 (T2|S1)', got '1 1 S2' ==> 1 1 (T2|S1|S2)
181 FAILED: Correct '1 1 C1', got '1 1 A1' ==> 1 1 (C1|A1)
201 FAILED: Correct '1 0 L2', got '1 0 M1' ==> 1 1 (M1|L2)


# File: STS-RV_Misc.tst

51 FAILED: Correct '1 1 (O14|N14|M14|K14|H12)', got '1 1 F11' => 1 1 (O14|N14|M14|K14|H12|F11)


# File STS-RV_RH.tst

7 FAILED: Correct '1 1 (F19|G19|H19)', got '1 1 E19' ==> 1 1 (F19|G19|H19|E19)
8 FAILED: Correct '1 1 (C19|D19|E19)', got '1 1 F19' ==> 1 1 (C19|D19|E19|F19)
16 FAILED: Correct '1 1 PASS', got '1 1 Q16' ==> 1 1 (PASS|Q16)
39 FAILED: Correct '1 1 Q1', got '1 1 R1' ==> 3 3 (Q1|R1)
40 FAILED: Correct '1 1 Q1', got '1 1 T2' ==> 3 3 (Q1|R1|T2)
53 FAILED: Correct '1 0 G14', got '1 0 J19' ==> 1 0 (G14|J19|J18|J17|H16)
60 FAILED: Correct '1 1 (G1|K1|D1)', got '1 1 J7' ==> 1 1 (G1|K1|D1|J7)
69 FAILED: Correct '1 1 N17', got '1 1 O19' ==> 1 1 (N17|O19)
78 FAILED: Correct '1 1 C12', got '1 1 C7' => 1 1 (C12|A7|B7|C7|D8|D9|D10|D11)
88 FAILED: Correct '1 1 (E18|G19|D16|G18|E19)', got '1 1 H19' ==> 1 1 (E18|G19|D16|G18|E19|H19)
90 FAILED: Correct '1 1 (H1|G1|E3|K5|L7)', got '1 1 J5' ==> 1 1 (H1|G1|E3|K5|L7|J5)


NOTE: The 39-40 is the unique testcase in wich I was forced 
to change the original result value (from [1 1] to [3 3])
I think a black stone in T2 is missed in diagram:
so if someone own the book of Richard Hunter, please 
checks this diagram (pag147_P05P08.sgf) and report it.

Also for test n. 78 I suspect a defect in diagram.