Many tests have results in negative form ([!...]), allowing any move except one or a few. This has turned out to often be problematic and a frequent source of "regression noise". Therefore most of those tests should be changed. Possible modifications are:
- Change to restricted_genmove when a specific problem should be addressed or when it's unclear what the globally best move is.
- Change to a positive result, when the globally acceptable moves can be determined and restricted_genmove is not more appropriate.
- Remove the test if there's not much point in it.
The current (GNU Go 3.7.8) number of negative test results for different test suites are
56 trevorc.tst
35 trevorb.tst
30 nngs3.tst
29 trevord.tst
22 trevor.tst
16 blunder.tst
12 trevora.tst
11 strategy4.tst
11 nngs4.tst
11 nngs1.tst
10 nngs2.tst
8 arend.tst
7 strategy3.tst
7 ninestones.tst
5 olympiad2004.tst
5 auto03.tst
4 handtalk.tst
4 century2002.tst
4 auto02.tst
3 strategy5.tst
3 strategy.tst
2 strategy2.tst
2 owl.tst
2 nngs.tst
2 nando.tst
2 manyfaces1.tst
2 dniwog.tst
2 buzco.tst
2 auto04.tst
1 semeai.tst
1 manyfaces.tst
1 gifu03.tst
1 ego.tst
1 auto01.tst
1 atari_atari.tst
1 arend2.tst
1 arb.tst
1 9x9.tst
1 13x13b.tst
for a total of 319.
To create this list, do
egrep -c '^#\? \[!' *.tst | grep -v ':0' | tr : ' ' | awk '{print $2 " " $1}' | sort -nr
