Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a tracing-based test harness for testing typing_copilot behavior #12

Open
obi1kenobi opened this issue Jul 15, 2021 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@obi1kenobi
Copy link
Owner

typing_copilot is already quite a complex tool, and is not trivial to test for correctness. To make it easier to test, split it into a "driver" which knows how to interact with mypy and the filesystem, and essentially a "library" that is fed data and is expected to act on it appropriately.

Then, different kinds of drivers could exist:

  • a driver that actually does read/write to disk and invoke mypy
  • a driver that wraps the one described above, but also records the actions taken and their outcomes into a trace file
  • a driver that reads a trace file and asserts that the library follows the trace as expected

The second driver could then be used to automatically generate traces of bug reports. The generated trace files could then be added to the test suite, and the third kind of driver could be used to ensure ongoing correctness in the face of the observed behaviors.

@obi1kenobi obi1kenobi added the enhancement New feature or request label Jul 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant