Change chess-python API to version 1.5 #19

karol-brejna-i · 2021-05-13T11:40:58Z

This PR upgrades python-chess package to the newest version and adds some basic regression tests.

vitogit · 2021-05-13T15:02:23Z

Pretty cool, awesome work @karol-brejna-i , I will try to review it this weekend as I want to take a better look.

vitogit · 2021-05-16T15:43:13Z

I run this with 3 games that in the previous version produced 1 or 2 tactics (I think is not deterministic) But this code produce 0 tactics
Here are the games

[Event "<1700 Rapid Arena"]
[Site "https://lichess.org/13Q4NqCj"]
[Date "2019.11.01"]
[White "PedrosF1"]
[Black "artemchampion"]
[Result "1-0"]
[UTCDate "2019.11.01"]
[UTCTime "08:47:32"]
[WhiteElo "1417"]
[BlackElo "1678"]
[WhiteRatingDiff "+12"]
[BlackRatingDiff "-11"]
[Variant "Standard"]
[TimeControl "600+0"]
[ECO "B40"]
[Termination "Normal"]

1. e4 c5 2. Nf3 e6 3. Nc3 a6 4. d4 b5 5. dxc5 Bxc5 6. Bd3 Qb6 7. O-O Bb7 8. Ne5 Nf6 9. a3 Nc6 10. Nxc6 Bxc6 11. b4 Bd4 12. Bd2 Nxe4 13. Bxe4 Bxe4 14. Nxe4 Bxa1 15. Qxa1 Rc8 16. Qxg7 Rf8 17. Nf6+ Ke7 18. Bg5 Kd6 19. Rd1+ Kc6 20. Nxd7 Rcd8 21. Bxd8 Rxd8 22. Qxf7 Qc7 23. Qxe6+ Kb7 24. Nc5+ Ka8 25. Rxd8+ Qxd8 26. Qc6+ Kb8 27. Qb7# 1-0

[Event "U1700 Rapid Arena"]
[Site "https://lichess.org/2BZx5TDD"]
[Date "2019.11.01"]
[Round "-"]
[White "Dicazman"]
[Black "Meju"]
[Result "0-1"]
[UTCDate "2019.11.01"]
[UTCTime "08:42:29"]
[WhiteElo "1547"]
[BlackElo "1629"]
[WhiteRatingDiff "-6"]
[BlackRatingDiff "+6"]
[Variant "Standard"]
[TimeControl "600+0"]
[ECO "C41"]
[Termination "Normal"]

1. e4 e5 2. Nf3 d6 3. d4 exd4 4. Nxd4 Nc6 5. Bb5 Bd7 6. Bxc6 Bxc6 7. Nc3 Nf6 8. Bg5 Be7 9. Bxf6 Bxf6 10. Nxc6 bxc6 11. O-O O-O 12. Re1 Be5 13. g3 Qd7 14. Rb1 Rfe8 15. Na4 Re6 16. Nc5 Qe7 17. Nxe6 Qxe6 18. b3 Qh3 19. Qf3 Re8 20. a4 c5 21. a5 Bc3 22. Re2 Bxa5 23. Ra1 Bb6 24. Qf5 Qh6 25. Qd7 Kf8 26. c4 Qh5 27. Re3 Qe5 28. Rd1 h5 29. f4 Qb2 30. f5 Qf6 31. e5 dxe5 32. Rxe5 Qxe5 33. Rf1 Qe3+ 34. Kh1 Qxb3 35. f6 g6 36. Qd5 c6 37. Qxc6 Qxc4 38. Qd6+ Kg8 39. Rd1 Qe4+ 40. Kg1 c4+ 41. Kf1 Qf3# 0-1

[Event "<1700 Rapid Arena"]
[Site "https://lichess.org/oFKr1UUx"]
[Date "2019.11.01"]
[White "fibberybosch"]
[Black "momentum"]
[Result "1-0"]
[UTCDate "2019.11.01"]
[UTCTime "08:41:29"]
[WhiteElo "1566"]
[BlackElo "1374"]
[WhiteRatingDiff "+4"]
[BlackRatingDiff "-3"]
[Variant "Standard"]
[TimeControl "600+0"]
[ECO "C21"]
[Termination "Normal"]

1. e4 e5 2. d4 exd4 3. c3 Nc6 4. cxd4 Bb4+ 5. Nc3 Bxc3+ 6. bxc3 d5 7. e5 Be6 8. Nf3 Nf6 9. exf6 gxf6 10. Bh6 Qe7 11. Be2 Bg4 12. O-O Bxf3 13. Bxf3 O-O-O 14. Rb1 Qd6 15. Qa4 a6 16. c4 dxc4 17. Qxc4 f5 18. d5 Ne5 19. Qf4 Nxf3+ 20. Qxf3 Qxh6 21. Rfc1 Rhg8 22. Qb3 b6 23. Qf3 Qg6 24. g3 h5 25. h4 Qg4 26. Qc3 Rd7 27. Rxb6 Rg6 28. Rb4 Qh3 29. Qh8+ Rd8 30. Rb8+ Kxb8 31. Qxd8+ Kb7 32. Rb1+ Rb6 33. Rc1 a5 34. Qxc7+ Ka6 35. Qc8+ Kb5 36. Rb1+ Ka4 37. Qc2+ Ka3 38. Qc3+ Ka4 39. Rxb6 Qg4 40. Qc2+ Ka3 41. Qb3# 1-0

Here are the tactics

[Event "<1700 Rapid Arena"]
[Site "https://lichess.org/13Q4NqCj"]
[Date "2019.11.01"]
[Round "?"]
[White "PedrosF1"]
[Black "artemchampion"]
[Result "1-0"]
[BlackElo "1678"]
[BlackRatingDiff "-11"]
[ECO "B40"]
[FEN "r3k2r/3p1ppp/pqb1p3/1p6/1P1bB3/P1N5/2PB1PPP/R2Q1RK1 b kq - 0 13"]
[SetUp "1"]
[Termination "Normal"]
[TimeControl "600+0"]
[UTCDate "2019.11.01"]
[UTCTime "08:47:32"]
[Variant "Standard"]
[WhiteElo "1417"]
[WhiteRatingDiff "+12"]

13... Bxe4 14. Nxe4 Bxa1 15. Qxa1 O-O 1-0

[Event "U1700 Rapid Arena"]
[Site "https://lichess.org/2BZx5TDD"]
[Date "2019.11.01"]
[Round "-"]
[White "Dicazman"]
[Black "Meju"]
[Result "0-1"]
[BlackElo "1629"]
[BlackRatingDiff "+6"]
[ECO "C41"]
[FEN "4r1k1/p1p2ppp/1b1p4/2p5/4P3/1P3QPq/2P1RP1P/R5K1 w - - 2 24"]
[SetUp "1"]
[Termination "Normal"]
[TimeControl "600+0"]
[UTCDate "2019.11.01"]
[UTCTime "08:42:29"]
[Variant "Standard"]
[WhiteElo "1547"]
[WhiteRatingDiff "-6"]

24. Qf5 Qxf5 25. exf5 Rxe2 26. c4 0-1

``

karol-brejna-i · 2021-05-17T07:47:44Z

I had similar experiences (the engine returned different scores between runs). I'll check the case.

karol-brejna-i · 2021-05-17T12:33:47Z

The engines produce a bit different scores so "investigate" doesn't count them as candidates for puzzles.
Maybe default engine settings in chess-python had changed... I'll check that.
I'll write some simple tests to check against the same scores, too.

karol-brejna-i · 2021-05-20T08:04:07Z

It is definitely sth with the new code. Will work on that and get back to you.

karol-brejna-i · 2021-05-31T10:57:18Z

I've created a branch (from master, https://github.com/vitogit/pgn-tactics-generator/compare/master...karol-brejna-i:producing-test-context?expand=1) to produce some test data on the older version.
I'll use them to validate the same functions in new versions.
Maybe when I finish the test we can decide if we want to include them (the tests) to the main codebase...

vitogit · 2021-05-31T13:42:29Z

I've created a branch (from master, https://github.com/vitogit/pgn-tactics-generator/compare/master...karol-brejna-i:producing-test-context?expand=1) to produce some test data on the older version.
I'll use them to validate the same functions in new versions.
Maybe when I finish the test we can decide if we want to include them (the tests) to the main codebase...

Given the nondeterministic nature of engines not sure how easy/hard would be to investigate it. One idea is to use single threading when running the engine --threads=1 to get more consistent results

karol-brejna-i · 2021-06-01T10:59:03Z

What I did was dump puzzle object (that contains analyzed positions including their scoring) to a json file.
I am using the file to recreate objects (without the need to use actual UCI engine) and then run some methods/functions, for example ambiguous(), is_complete().

@vitogit could you take a look at the branch https://github.com/karol-brejna-i/pgn-tactics-generator/tree/producing-test-context and the following lines of ambiguous method: https://github.com/vitogit/pgn-tactics-generator/blob/master/modules/puzzle/position_list.py#L146-L147

            if self.analysed_legals[0].evaluation.mate is None or self.analysed_legals[1].evaluation.cp is None:
                return

This method is supposed to return a boolean value. These lines return nothing.
(This is one of the reasons the results differ -- upgraded version vs current version -- but not the only one, unfortunately).

I assume that the intent was to return True here. Could you confirm?

vitogit · 2021-06-01T20:42:12Z

What I did was dump puzzle object (that contains analyzed positions including their scoring) to a json file.
I am using the file to recreate objects (without the need to use actual UCI engine) and then run some methods/functions, for example ambiguous(), is_complete().

@vitogit could you take a look at the branch https://github.com/karol-brejna-i/pgn-tactics-generator/tree/producing-test-context and the following lines of ambiguous method: https://github.com/vitogit/pgn-tactics-generator/blob/master/modules/puzzle/position_list.py#L146-L147
            if self.analysed_legals[0].evaluation.mate is None or self.analysed_legals[1].evaluation.cp is None:
                return
This method is supposed to return a boolean value. These lines return nothing.
(This is one of the reasons the results differ -- upgraded version vs current version -- but not the only one, unfortunately).

I assume that the intent was to return True here. Could you confirm?

That line was added in your previous PR 23f543c#diff-e31f1011e582ea3d502823f13a4a64fdef87e41ed47178a0a6e497df1402a27cR146
Probably that whole function can be refactored but looking at the previous code I think you want to return false because you inverted the if from

self.analysed_legals[0].evaluation.mate is not None and self.analysed_legals[1].evaluation.cp is not None

to

self.analysed_legals[0].evaluation.mate is None or self.analysed_legals[1].evaluation.cp is None:

or leave it as it was before

karol-brejna-i · 2021-06-02T13:08:56Z

Jeez, it looks like you are right. I broke this. Let me fix this first with a PR.

karol-brejna-i · 2021-06-02T13:17:59Z

I've created #24 to restore the condition.

Going back to test this evening.

karol-brejna-i · 2021-06-03T06:52:20Z

With ambiguous fixed, the code works as expected. Regression tests that I added confirm that ambiguous(), is_complete() and investigate() return the same results in both versions. The games are produced similarly.
@vitogit Please, take a look.

modules/decoding.py

Restructure to allow running the test from the commandline; Fix broken ambiguous() condition

…pytest'

karol-brejna-i · 2021-06-07T12:27:56Z

I made some cleanups. Looks like this one is finally ready to be merged.
@vitogit , please, take a look!

modules/puzzle/position_list.py

positions_for_investigation.py

vitogit · 2021-06-07T18:04:21Z

test/unit/test_regression.py

@@ -0,0 +1,102 @@
+import json


This is great. What do you think about adding a small section in the readme with some info about the tests, how to run them, etc

I was thinking about a dedicated document about development aspect (how to build and publish a package -- in the future, how to test, etc.) This way we'd had a readme that is all about usage of the software, and other that is focused on development/testing. I'll start one today and share.

take a look at https://github.com/karol-brejna-i/pgn-tactics-generator/blob/chess-python-upgrade/DEVELOPMENT.md

Excellent I like it.

vitogit · 2021-06-09T14:17:30Z

DEVELOPMENT.md

+
+The recommended way to run the test is using `pytest`:
+```bash
+pytest


This is not in the requirements. Maybe add a line about how to install it

pip3 install -U pytest

DEVELOPMENT.md

vitogit

@karol-brejna-i I just left a couple of simple comments but is approved. I added you as a collaborator, so you can merge it when you want. Thank you for this awesome PR and all the improvements!

* upstream/master: Merge pull request vitogit#19; use python-chess 1.5

karol-brejna-i force-pushed the chess-python-upgrade branch from d55fdd4 to ef41509 Compare May 13, 2021 12:25

karol-brejna-i changed the title ~~Change chess-python API to version 1.5~~ [WiP] Change chess-python API to version 1.5 May 20, 2021

karol-brejna-i changed the title ~~[WiP] Change chess-python API to version 1.5~~ Change chess-python API to version 1.5 Jun 3, 2021

vitogit reviewed Jun 5, 2021

View reviewed changes

modules/decoding.py Outdated Show resolved Hide resolved

karol-brejna-i changed the title ~~Change chess-python API to version 1.5~~ [WiP] Change chess-python API to version 1.5 Jun 7, 2021

karol-brejna-i added 3 commits June 7, 2021 12:46

Change chess-python API to v1.5;

91c94b0

Restructure to allow running the test from the commandline; Fix broken ambiguous() condition

Improve evaluation encoding; Making tests run from commandline wity '…

662e12a

…pytest'

Remove references to chess.uci; add test data for Score(0)

f26bdda

karol-brejna-i force-pushed the chess-python-upgrade branch from 76482aa to f26bdda Compare June 7, 2021 12:25

karol-brejna-i changed the title ~~[WiP] Change chess-python API to version 1.5~~ Change chess-python API to version 1.5 Jun 7, 2021

vitogit reviewed Jun 7, 2021

View reviewed changes

modules/puzzle/position_list.py Show resolved Hide resolved

vitogit reviewed Jun 7, 2021

View reviewed changes

positions_for_investigation.py Outdated Show resolved Hide resolved

vitogit reviewed Jun 7, 2021

View reviewed changes

Add info on running the tests

d65f659

vitogit reviewed Jun 9, 2021

View reviewed changes

DEVELOPMENT.md Show resolved Hide resolved

vitogit approved these changes Jun 9, 2021

View reviewed changes

karol-brejna-i merged commit a3b9195 into vitogit:master Jun 10, 2021

pnodet added a commit to pnodet/pgn-tactics-generator that referenced this pull request Jul 20, 2021

Merge remote-tracking branch 'upstream/master'

5ef7b7e

* upstream/master: Merge pull request vitogit#19; use python-chess 1.5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change chess-python API to version 1.5 #19

Change chess-python API to version 1.5 #19

karol-brejna-i commented May 13, 2021 •

edited

Loading

vitogit commented May 13, 2021

vitogit commented May 16, 2021

karol-brejna-i commented May 17, 2021

karol-brejna-i commented May 17, 2021

karol-brejna-i commented May 20, 2021

karol-brejna-i commented May 31, 2021

vitogit commented May 31, 2021

karol-brejna-i commented Jun 1, 2021

vitogit commented Jun 1, 2021

karol-brejna-i commented Jun 2, 2021

karol-brejna-i commented Jun 2, 2021

karol-brejna-i commented Jun 3, 2021 •

edited

Loading

karol-brejna-i commented Jun 7, 2021

vitogit Jun 7, 2021

karol-brejna-i Jun 8, 2021

karol-brejna-i Jun 8, 2021

vitogit Jun 9, 2021

vitogit Jun 9, 2021

vitogit left a comment

Change chess-python API to version 1.5 #19

Change chess-python API to version 1.5 #19

Conversation

karol-brejna-i commented May 13, 2021 • edited Loading

vitogit commented May 13, 2021

vitogit commented May 16, 2021

karol-brejna-i commented May 17, 2021

karol-brejna-i commented May 17, 2021

karol-brejna-i commented May 20, 2021

karol-brejna-i commented May 31, 2021

vitogit commented May 31, 2021

karol-brejna-i commented Jun 1, 2021

vitogit commented Jun 1, 2021

karol-brejna-i commented Jun 2, 2021

karol-brejna-i commented Jun 2, 2021

karol-brejna-i commented Jun 3, 2021 • edited Loading

karol-brejna-i commented Jun 7, 2021

vitogit Jun 7, 2021

Choose a reason for hiding this comment

karol-brejna-i Jun 8, 2021

Choose a reason for hiding this comment

karol-brejna-i Jun 8, 2021

Choose a reason for hiding this comment

vitogit Jun 9, 2021

Choose a reason for hiding this comment

vitogit Jun 9, 2021

Choose a reason for hiding this comment

vitogit left a comment

Choose a reason for hiding this comment

karol-brejna-i commented May 13, 2021 •

edited

Loading

karol-brejna-i commented Jun 3, 2021 •

edited

Loading