Cannot run individual tests when using "pytest"

[jpgxs]

Description of the feature request:

When using pytest, there are no play icons ▶️ for running individual tests (individual methods, classes, modules) using Bazel.

It appears that only unittest.TestCase is supported, based on python/src/com/google/idea/blaze/python/run/PyTestUtils.java.

I'm happy to submit a PR for this if we can agree on the implementation.

Which category does this issue belong to?

Intellij, PyCharm

What underlying problem are you trying to solve with this feature?

Want to run run filtered pytest tests from the IntelliJ/PyCharm editor

What operating system, Intellij IDE and programming languages are you using? Please provide specific versions.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[jpgxs]

To add support for pytest, we would need to detect:

a. Test Classes: any class with name "Test*"
b. Test Methods: any method with name "test_" (possibly inside a test class)
c. (for bonus points) Test Modules: any module with name "test_"

(a) and (b) are relatively easy - not so sure about (c) though.

In any case, we would need to agree...

• Should this be an opt-in feature? If so, how should it be opted into and where should it be documented?
• Should we use the "Python Integrated Tools -> Default test runner" setting?

[JeroenSchmidt]

Hi @jpgxs
I'm a user of the plugin and I'm able to select individual pytest tests to run. Debugging also works. See the screenshot.
I would suggest you check that your using the latest version of pycharm/intelij (+python plugin)

Having said that. When running an "individual test" it ends up running all test tests in that target (I'm not sure if this is expected behaviour or a bug). A question to the maintainers, is there a reason something like the following isn't passed? bazel test //path/to/target:test --test_arg='-k {test_function_name}'

[jpgxs]

Thanks @JeroenSchmidt, I seem to get some interesting behaviour now that I'm looking closely.

If I define a test at the module level (not inside a class), I get the same behaviour as you:

However, for test classes (Test*.py) and their methods (test_*.py), I either get:

1. No play button
2. A play button with "Nothing here"
3. A play button to run all the tests in the Bazel test

It's not clear to me why (1) happens as it seems to have changed over the course of today; I think it's related to IntelliJ's native pytest integration + caching.

Currently, I get "Nothing Here" for all tests in the file:

This changes in all cases if I either:

(a) Define a test at the module level:

(b) Define a test class with name "*Test":

I think this logic is being handled here:

intellij/python/src/com/google/idea/blaze/python/run/PyTestUtils.java

Line 26 in 76f1972

public class PyTestUtils {

---

Of course, these aren't very useful if they run all the tests in the suite anyway; some of our py_test targets take 10+ minutes to run if you're running everything!

The unittest integration seems to pass though the --test_filter argument with TestClass.test_method / TestClass / test_method.

For pytest, I think it should either:

a. Pass it through with exactly the same scheme as unittest.
b. Pass it through in a pytest native format module.py::TestClass::test_method

Personally, I'm leaning towards (b) but it might be a bit trickier

[tpasternak]

I'm happy to submit a PR for this if we can agree on the implementation.

Thank you @jpgxs , you're welcome! cc @agluszak