Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_dir feature for resolver #5576

Open
richtja opened this issue Jan 12, 2023 · 0 comments
Open

test_dir feature for resolver #5576

richtja opened this issue Jan 12, 2023 · 0 comments

Comments

@richtja
Copy link
Contributor

richtja commented Jan 12, 2023

Is your feature request related to a problem? Please describe.
On the legacy runner was different approach in listing avocado tests. The avocado had a default test directory under datadir.paths.test_dir variable and it was possible to run avocado list command to list all tests from this default directory. When we moved from the legacy architecture to the resolver architecture, we removed this functionality. Now users have to specify the path to the test directory in the avocado command like this avocado list examples/tests/

When you run many different sets of tests in one avocado job, it will save time to specific the path to the test directory where the tests are stored. This feature will bring back the datadir.paths.test_dir feature from the legacy architecture.

Describe the solution you'd like

$ cat /root/.config/avocado/avocado.conf
[datadir.paths]
test_dir = /avocado_dir/examples/tests

$ avocado list
avocado-instrumented examples/tests/abort.py:AbortTest.test
avocado-instrumented examples/tests/assert.py:Assert.test_assert_raises
avocado-instrumented examples/tests/assert.py:Assert.test_fails_to_raise
avocado-instrumented examples/tests/assets.py:Hello.test_gpg_signature
avocado-instrumented examples/tests/assets.py:Hello.test_build_run
avocado-instrumented examples/tests/cabort.py:CAbort.test
avocado-instrumented examples/tests/cancel_on_exception.py:CancelOnException.test
avocado-instrumented examples/tests/cancel_test.py:CancelTest.test_iperf
avocado-instrumented examples/tests/cancel_test.py:CancelTest.test_gcc
avocado-instrumented examples/tests/cancelonsetup.py:CancelOnSetupTest.test_wont_be_executed
avocado-instrumented examples/tests/canceltest.py:CancelTest.test
avocado-instrumented examples/tests/cit_parameters.py:CitParameters.test
exec-test            examples/tests/custom_env_variable.sh
avocado-instrumented examples/tests/datadir.py:DataDirTest.test
avocado-instrumented examples/tests/dependency_ansible.py:FileByAnsible.test
...

Job API usage:

$ cat job.py
#!/usr/bin/env python3

import sys

from avocado.core.job import Job
from avocado.core.suite import TestSuite

job_config = {
    "datadir.paths.test_dir": "examples/tests",
}

suite_pass_config = {
    "resolver.references": ["/bin/true", "passtest.py"],
}

suite_fail_config = {
    "resolver.references": ["failtest.py"],
}

with Job(
    config=job_config,
    test_suites=[
        TestSuite.from_config(suite_pass_config, name="pass"),
        TestSuite.from_config(suite_fail_config, name="fail"),
    ]
) as j:
    sys.exit(j.run())

$ python job.py
JOB ID     : af6e3bf63665ecf3a9d1e1b3a1111fe66ec7b76e
JOB LOG    : /home/janrichter/avocado/job-results/job-2023-01-12T16.00-af6e3bf/job.log
 (pass-2/2) examples/tests/passtest.py:PassTest.test: STARTED
 (pass-1/2) /bin/true: STARTED
 (pass-1/2) /bin/true: PASS (0.01 s)
 (pass-2/2) examples/tests/passtest.py:PassTest.test: PASS (0.02 s)
 (fail-1/1) examples/tests/failtest.py:FailTest.test: STARTED
 (fail-1/1) examples/tests/failtest.py:FailTest.test: FAIL: This test is supposed to fail (0.04 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /home/janrichter/avocado/job-results/job-2023-01-12T16.00-af6e3bf/results.html
JOB TIME   : 2.56 s

Additional information
The legacy datadir.paths.test_dir variable still has some leftovers in code and documentation. Part of this issue should be removed unnecessary legacy code of test_dir feature.

This issue is based on #5571

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Long Term (Next Q) Backlog
Development

No branches or pull requests

1 participant