Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(testretry): Add testretry command #11

Merged
merged 3 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,73 @@
# kci-dev

kci-dev is a cmdline tool for interact with a enabled KernelCI server
Purpose of this tool to provide a easy way to use features of KernelCI Pipeline instance.

## Installation

Using poetry and virtualenv
```sh
virtualenv .venv
source .venv/bin/activate
pip install poetry
poetry install
poetry run kci-dev
```

## Configuration

kci-dev uses a configuration file .kci-dev.toml in the program directory.
```toml
default_instance="staging"
[local]
host="https://127.0.0.1"
token="example"

[staging]
host="https://staging.kernelci.org:9100/"
token="SOMEVERYSECRETTOKEN"

[production]
host="https://kernelci-pipeline.westus3.cloudapp.azure.com/"
token="example"
```

Where `default_instance` is the default instance to use, if not provided in the command line.
In section `local`, `staging`, `production` you can provide the host and token for the available instances.
host is the URL of the KernelCI Pipeline API endpoint, and token is the API token to use for authentication.
If you are using KernelCI Pipeline instance, you can get the token from the project maintainers.
If it is a local instance, you can generate your token using kernelci-pipeline/tools/jwt_generator.py script.

## Options

### instance
You can provide the instance name to use for the command.

Example:
```sh
kci-dev --instance staging
```

### settings

You can provide the configuration file path to use for the command.

Example:
```sh
kci-dev --settings /path/to/.kci-dev.toml
```

## Commands

### testretry

This command will retry the failed tests. In some cases tests may fail due to network issues, hardware problems,
nature of test (flaky), etc. This command will retry the failed tests, and create additional test jobs for the failed tests.
After observing the results, you can decide if test results were reliable, not, or maybe even test need improvement.

Example:
```sh
kci-dev testretry --nodeid <testnodeid>
```

testnodeid is the node id of the test job, which you can get from the KernelCI dashboard. Usually it is hexadecimal string.
3 changes: 2 additions & 1 deletion kci-dev/kci-dev.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

import click
from libs.common import *
from subcommands import commit, patch, results
from subcommands import commit, patch, results, testretry


@click.group(
Expand Down Expand Up @@ -32,6 +32,7 @@ def run():
cli.add_command(commit.commit)
cli.add_command(patch.patch)
cli.add_command(results.results)
cli.add_command(testretry.testretry)
cli()


Expand Down
63 changes: 63 additions & 0 deletions kci-dev/subcommands/testretry.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import json

import click
import requests
from git import Repo
from libs.common import *


def api_connection(host):
click.secho("api connect: " + host, fg="green")
return host


def display_api_error(response):
click.secho(f"API response error code: {response.status_code}", fg="red")
try:
click.secho(response.json(), fg="red")
except json.decoder.JSONDecodeError:
click.secho(f"No JSON response. Plain text: {response.text}", fg="yellow")
return


def send_jobretry(baseurl, jobid, token):
url = baseurl + "api/jobretry"
headers = {
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"{token}",
}
data = {"nodeid": jobid}
jdata = json.dumps(data)
try:
response = requests.post(url, headers=headers, data=jdata)
except requests.exceptions.RequestException as e:
click.secho(f"API connection error: {e}", fg="red")
return

if response.status_code != 200:
display_api_error(response)
return None
return response.json()


@click.command(help="Retry a test(job) on KernelCI")
@click.option(
"--nodeid",
help="define the node id of the (test)job to retry",
required=True,
)
@click.pass_context
def testretry(ctx, nodeid):
cfg = ctx.obj.get("CFG")
instance = ctx.obj.get("INSTANCE")
url = api_connection(cfg[instance]["host"])
resp = send_jobretry(url, nodeid, cfg[instance]["token"])
if resp and "message" in resp:
click.secho(resp["message"], fg="green")


if __name__ == "__main__":
main_kcidev()
11 changes: 11 additions & 0 deletions tests/test_kcidev.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,17 @@ def test_kcidev_results_help():
assert result.returncode == 0


def test_kcidev_testretry_help():
command = ["poetry", "run", "kci-dev", "testretry", "--help"]
result = run(command, stdout=PIPE, stderr=PIPE, universal_newlines=True)
print("returncode: " + str(result.returncode))
print("#### stdout ####")
print(result.stdout)
print("#### stderr ####")
print(result.stderr)
assert result.returncode == 0


def test_kcidev_results_tests():
command = [
"poetry",
Expand Down
Loading