Skip to content

Implement command to validate builds #170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

JenySadadia
Copy link

@JenySadadia JenySadadia commented Jun 5, 2025

Closes #171

Get builds from dashboard and validate it with maestro result.
PR created on top of #177 and #179
Depends on kernelci/kernelci-api#615

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from c8c00e6 to 99f9ec1 Compare June 5, 2025 09:46
@padovan
Copy link
Contributor

padovan commented Jun 5, 2025

I am getting this when running the command without arguments:

kci-dev > poetry run kci-dev validate-builds
Validate builds called----
git folder: None
tree: https://github.com/kernelci/kci-dev.git
branch: validate-builds-cmd
commit: 99f9ec1a74f6d6817feae8902f627187a997cdf6
json error: Tree checkout not found
Aborted!

Guess we need to throw a clear error message guiding the user.

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from 99f9ec1 to 6d8d4a9 Compare June 6, 2025 09:30
@JenySadadia
Copy link
Author

I am getting this when running the command without arguments:

kci-dev > poetry run kci-dev validate-builds
Validate builds called----
git folder: None
tree: https://github.com/kernelci/kci-dev.git
branch: validate-builds-cmd
commit: 99f9ec1a74f6d6817feae8902f627187a997cdf6
json error: Tree checkout not found
Aborted!

Guess we need to throw a clear error message guiding the user.

Thanks. Addressed this.

@JenySadadia
Copy link
Author

JenySadadia commented Jun 6, 2025

For starters, the command will work by providing git URL, branch, and commit.
Here are the samples:

$ kci-dev --settings ./kci-dev.toml validate-builds --giturl https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git --branch master --commit 7d4e49a77d9930c69751b9192448fda6ff9100f1 --arch arm
Maestro:
Total nodes found in maestro:  15
Dashboard:
15
Build count matched
$ kci-dev --settings ./kci-dev.toml validate-builds --commit fb01b86bf35a2aecfb4e777c1cb81a69203edc0b --giturl https://github.com/kernelci/linux.git --branch staging-next
Maestro:
Total nodes found in maestro:  0
Dashboard:
json error: Tree checkout not found
Aborted!
$ kci-dev --settings ./kci-dev.toml validate-builds --commit 8b5d78fb5c56fd82d8e621644ff073a375fa83d6 --giturl https://android.googlesource.com/kernel/common --branch android15-6.6
Maestro:
Total nodes found in maestro:  0
Dashboard:
json error: No builds found for this tree checkout
Aborted!

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from 6d8d4a9 to faef030 Compare June 10, 2025 11:31
@JenySadadia JenySadadia marked this pull request as ready for review June 10, 2025 11:31
@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from faef030 to 54c647f Compare June 10, 2025 11:33
@JenySadadia
Copy link
Author

JenySadadia commented Jun 10, 2025

Modified directory structure, refactored the code and commands.
The new commands are as below:

  1. kci-dev validate build all-checkouts
  2. kci-dev validate build specific-checkout --giturl <URL> --commit <commit-hash> --branch <git-branch>

Both commands will provide below information:

  1. Build count validation
  2. Missing build information
  3. Build status validation

Sample result:

$ kci-dev --settings ./kci-dev.toml validate build specific-checkout --giturl https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git --branch master --commit 7d4e49a77d9930c69751b9192448fda6ff9100f1 
Maestro:
Total nodes found in maestro:  45
Dashboard:
45
Build count matched
$ kci-dev --settings ./kci-dev.toml validate build all-checkouts
URL: https://chromium.googlesource.com/chromiumos/third_party/kernel.git
branch: chromeos-6.1
commit: e05128ed2715843e6e03ce05d20ca32cb82a493e
Maestro:
Total nodes found in maestro:  5
Dashboard:
4
Missing builds: [{'id': '683f98f5214050d3418d0917', 'kind': 'kbuild', 'name': 'kbuild-clang-17-x86-chromeos-daily-intel', 'path': ['checkout', 'kbuild-clang-17-x86-chromeos-daily-intel'], 'group': 'kbuild-clang-17-x86-chromeos-daily-intel', 'parent': '683f9836214050d3418d03a5', 'state': 'done', 'result': 'incomplete', 'artifacts': {}, 'data': {'kernel_revision': {'tree': 'chromiumos', 'url': 'https://chromium.googlesource.com/chromiumos/third_party/kernel.git', 'branch': 'chromeos-6.1', 'commit': 'e05128ed2715843e6e03ce05d20ca32cb82a493e', 'describe': 'v6.1.129-17451-ge05128ed27158', 'version': {'version': 6, 'patchlevel': 1, 'sublevel': 129, 'extra': '-17451-ge05128ed27158'}, 'commit_tags': [], 'commit_message': 'CHROMIUM: config: Enable CONFIG_USB_LINK_LAYER_TEST for MTK\n\nEnable CONFIG_USB_LINK_LAYER_TEST as a module to add the lvstest\ndriver.\n\nBUG=b:412565469\nTEST=CQ, tested on 6.6 kernel\n\nChange-Id: Icbdc91eb35253a925a1a03bbadffd87f7e89c2c2\nSigned-off-by: Jameson Thies <[email protected]>\nReviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/6594756\nReviewed-by: Abe Levkoy <[email protected]>', 'tip_of_branch': True}, 'runtime': 'k8s-all', 'platform': 'kubernetes', 'job_id': 'kci-683f98f5214050d3418d0917-kbuild-clang-17-x86-chro-t45qnxb0', 'job_context': 'aks-kbuild-medium-1', 'error_code': 'node_timeout', 'error_msg': 'Node timed-out'}, 'debug': {}, 'jobfilter': None, 'platform_filter': [], 'created': '2025-06-04T00:53:09.138000', 'updated': '2025-06-04T07:02:53.365000', 'timeout': '2025-06-04T06:53:09.138000', 'holdoff': None, 'owner': 'production', 'submitter': 'service:pipeline', 'treeid': '8e05534a68f0a8d700c47ec16323ebab2f5d25a3eda2f4fbcacb97f6c85da43a', 'user_groups': [], 'processed_by_kcidb_bridge': True}]
Build count didn't match

Note: The all-checkouts command prints available trees list at the beginning.
Need to hide it from the output.

@padovan
Copy link
Contributor

padovan commented Jun 10, 2025

Thanks for this, @JenySadadia.

As for the overall command structure. I have been thinking that we need to create kci-dev maestro, so we can move a lot of maestro specific commands under it. (eg kci-dev maestro-results should become kci-dev maestro results).

So I suggest:

  1. A small detour to start ``kci-dev maestroaddingkci-dev maestro results` under it.
  2. Make this patchset use that: kci-dev maestro validate

Now looking at the options you added, my suggestion would be:

kci-dev maestro validate build --all-checkouts
kci-dev maestro validate build --giturl <URL> --commit <commit-hash> --branch <git-branch>

Let's also make the output a bit more standard:

Maestro:  45
Dashboard: 45
Build count matched

@aliceinwire
Copy link
Member

aliceinwire commented Jun 11, 2025

When using specific-checkout the command should auto fill missing parameters, in a similar fashion of kci-dev results summary for example

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from 54c647f to aa67aa2 Compare June 11, 2025 07:39
@JenySadadia
Copy link
Author

Thanks for this, @JenySadadia.

As for the overall command structure. I have been thinking that we need to create kci-dev maestro, so we can move a lot of maestro specific commands under it. (eg kci-dev maestro-results should become kci-dev maestro results).

So I suggest:

  1. A small detour to start ``kci-dev maestroaddingkci-dev maestro results` under it.
  2. Make this patchset use that: kci-dev maestro validate

Yes, makes sense.
I'll create another PR for this as this is out of scope for the current PR.

Now looking at the options you added, my suggestion would be:

kci-dev maestro validate build --all-checkouts kci-dev maestro validate build --giturl <URL> --commit <commit-hash> --branch <git-branch>

Let's also make the output a bit more standard:

Maestro:  45
Dashboard: 45
Build count matched

Thanks for the suggestion. Addressed this.
The output looks like the below now:

Maestro:
Build count: 45
Dashboard:
Build count:45
Build count matched

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch 3 times, most recently from 873f2ae to bb20725 Compare June 11, 2025 08:54
@JenySadadia
Copy link
Author

When using specific-checkout the command should auto fill missing parameters, in a similar fashion of kci-dev results summary for example

Thanks for the comment. Addressed this.

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from bb20725 to 55b5f33 Compare June 11, 2025 08:56
@padovan
Copy link
Contributor

padovan commented Jun 11, 2025

When using specific-checkout the command should auto fill missing parameters, in a similar fashion of kci-dev results summary for example

That is not so important for this usecase. validate-builds will be more a of backend validation tool for maestro, so not the usual kernel dev doing his development.

@aliceinwire
Copy link
Member

aliceinwire commented Jun 11, 2025

When using specific-checkout the command should auto fill missing parameters, in a similar fashion of kci-dev results summary for example

That is not so important for this usecase. validate-builds will be more a of backend validation tool for maestro, so not the usual kernel dev doing his development.

I had found it useful when testing to get the information directly from the current folder instead of adding it by hand each time

@aliceinwire
Copy link
Member

aliceinwire commented Jun 12, 2025

@padovan Are we supposed to see Missing builds: as a Json dictionary string ? or we can show it in a better way?

Missing builds: [{'id': '683f98f5214050d3418d0917', 'kind': 'kbuild', 'name': 'kbuild-clang-17-x86-chromeos-daily-intel', 'path': ['checkout', 'kbuild-clang-17-x86-chromeos-daily-intel'], 'group': 'kbuild-clang-17-x86-chromeos-daily-intel', 'parent': '683f9836214050d3418d03a5', 'state': 'done', 'result': 'incomplete', 'artifacts': {}, 'data': {'kernel_revision': {'tree': 'chromiumos', 'url': 'https://chromium.googlesource.com/chromiumos/third_party/kernel.git', 'branch': 'chromeos-6.1', 'commit': 'e05128ed2715843e6e03ce05d20ca32cb82a493e', 'describe': 'v6.1.129-17451-ge05128ed27158', 'version': {'version': 6, 'patchlevel': 1, 'sublevel': 129, 'extra': '-17451-ge05128ed27158'}, 'commit_tags': [], 'commit_message': 'CHROMIUM: config: Enable CONFIG_USB_LINK_LAYER_TEST for MTK\n\nEnable CONFIG_USB_LINK_LAYER_TEST as a module to add the lvstest\ndriver.\n\nBUG=b:412565469\nTEST=CQ, tested on 6.6 kernel\n\nChange-Id: Icbdc91eb35253a925a1a03bbadffd87f7e89c2c2\nSigned-off-by: Jameson Thies <[email protected]>\nReviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/6594756\nReviewed-by: Abe Levkoy <[email protected]>', 'tip_of_branch': True}, 'runtime': 'k8s-all', 'platform': 'kubernetes', 'job_id': 'kci-683f98f5214050d3418d0917-kbuild-clang-17-x86-chro-t45qnxb0', 'job_context': 'aks-kbuild-medium-1', 'error_code': 'node_timeout', 'error_msg': 'Node timed-out'}, 'debug': {}, 'jobfilter': None, 'platform_filter': [], 'created': '2025-06-04T00:53:09.138000', 'updated': '2025-06-04T07:02:53.365000', 'timeout': '2025-06-04T06:53:09.138000', 'holdoff': None, 'owner': 'production', 'submitter': 'service:pipeline', 'treeid': '8e05534a68f0a8d700c47ec16323ebab2f5d25a3eda2f4fbcacb97f6c85da43a', 'user_groups': [], 'processed_by_kcidb_bridge': True}]

@JenySadadia
Copy link
Author

@aliceinwire I reformatted output to be nicer:

$ kci-dev --settings ./kci-dev.toml validate build --giturl https://android.googlesource.com/kernel/common --commit de3036f041b742dfd2b0d86fc50ce61aaeca0aa1 --branch android12-5.4
Maestro:
Build count:34
Dashboard:
Build count:33
Build count didn't match
[
 {
  "id": "6849fb29dba9af5ad62c489b",
  "kind": "kbuild",
  "name": "kbuild-clang-17-x86-android-allmodconfig",
  "path": [
   "checkout",
   "kbuild-clang-17-x86-android-allmodconfig"
  ],
  "group": "kbuild-clang-17-x86-android-allmodconfig",
  "parent": "6849f8c8dba9af5ad62c4753",
  "state": "done",
  "result": "incomplete",
  "artifacts": {},
  "data": {
   "kernel_revision": {
    "tree": "android",
    "url": "https://android.googlesource.com/kernel/common",
    "branch": "android12-5.4",
    "commit": "de3036f041b742dfd2b0d86fc50ce61aaeca0aa1",
    "describe": "ASB-2025-06-05_12-5.4-2-gde3036f041b74",
    "version": {
     "version": 5,
     "patchlevel": 4,
     "sublevel": 292,
     "extra": "-5245-gde3036f041b74"
    },
    "commit_tags": [],
    "commit_message": "BACKPORT: ...",
    "tip_of_branch": true
   },
   "runtime": "k8s-all",
   "platform": "kubernetes",
   "job_id": "kci-6849fb29dba9af5ad62c489b-kbuild-clang-17-x86-andr-3xsdj1ze",
   "job_context": "gke_android-kernelci-external_us-east4-c_kci-big-us-east4",
   "error_code": "node_timeout",
   "error_msg": "Node timed-out"
  },
  "debug": {},
  "jobfilter": null,
  "platform_filter": [],
  "created": "2025-06-11T21:54:49.223000",
  "updated": "2025-06-12T04:00:58.200000",
  "timeout": "2025-06-12T03:54:49.223000",
  "holdoff": null,
  "owner": "production",
  "submitter": "service:pipeline",
  "treeid": "292f74a2cd242808aec687fdde0069e435a860ff4e00c53ed86cbbb0fb793ece",
  "user_groups": [],
  "processed_by_kcidb_bridge": true
 }
]

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch 5 times, most recently from 67b1310 to 8ea4f82 Compare June 12, 2025 10:01
@padovan
Copy link
Contributor

padovan commented Jun 12, 2025

@aliceinwire I reformatted output to be nicer:

$ kci-dev --settings ./kci-dev.toml validate build --giturl https://android.googlesource.com/kernel/common --commit de3036f041b742dfd2b0d86fc50ce61aaeca0aa1 --branch android12-5.4
Maestro:
Build count:34
Dashboard:
Build count:33
Build count didn't match
[
 {
  "id": "6849fb29dba9af5ad62c489b",
  "kind": "kbuild",
  "name": "kbuild-clang-17-x86-android-allmodconfig",
 ...

I believe that just putting a list of ids would be enough, or a list of id + link to our maestro api viewer, otherwise the output will be quite verbose.

@aliceinwire
Copy link
Member

@aliceinwire I reformatted output to be nicer:

$ kci-dev --settings ./kci-dev.toml validate build --giturl https://android.googlesource.com/kernel/common --commit de3036f041b742dfd2b0d86fc50ce61aaeca0aa1 --branch android12-5.4
Maestro:
Build count:34
Dashboard:
Build count:33
Build count didn't match
[
 {
  "id": "6849fb29dba9af5ad62c489b",
  "kind": "kbuild",
  "name": "kbuild-clang-17-x86-android-allmodconfig",
 ...

I believe that just putting a list of ids would be enough, or a list of id + link to our maestro api viewer, otherwise the output will be quite verbose.

Would be good to have by default a list of ids and have a --verbose argument for having the full verbose output

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from 8ea4f82 to 6b883f1 Compare June 13, 2025 06:53
@JenySadadia
Copy link
Author

Would be good to have by default a list of ids and have a --verbose argument for having the full verbose output

Thanks for the suggestion. Added --verbose option.

@aliceinwire
Copy link
Member

you added the commit Enable --days switch for getting tree listEnable --days switch for getting tree list of a different PR on this PR

@JenySadadia
Copy link
Author

you added the commit Enable --days switch for getting tree listEnable --days switch for getting tree list of a different PR on this PR

Yes, this PR depends on 2 other PRs.
Otherwise, the command won't work properly. See #170 (comment)

@padovan
Copy link
Contributor

padovan commented Jun 13, 2025

Thanks for the PR @JenySadadia. I found a few issues:

  1. It seems the code is trying to look for a local git tree even if I'm supplying --giturl:
poutine kci-dev > poetry run kci-dev validate build --giturl https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git --branch master --commit 7d4e49a77d9930c69751b9192448fda6ff9100f1 
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
branch: master
commit: 7d4e49a77d9930c69751b9192448fda6ff9100f1
Maestro:
Build count:45
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
branch: master
commit: 7d4e49a77d9930c69751b9192448fda6ff9100f1
Dashboard:
Build count:45
Build count matched
  1. if no commands are provided and the git tree doesn't exist in the dashboard, we should fail much cleaner than this:
poutine kci-dev > poetry run kci-dev validate build
git folder: None
tree: https://github.com/kernelci/kci-dev.git
branch: validate-builds-cmd
commit: 6b883f19e50f5750f3aa8757b462f6008658b017
Maestro:
Build count:0
git folder: None
tree: https://github.com/kernelci/kci-dev.git
branch: validate-builds-cmd
commit: 6b883f19e50f5750f3aa8757b462f6008658b017
json error: No results available for this tree/branch/commit
Aborted while fetching dashboard builds
Build count matched
  1. all checkouts out too verbose:
URL: https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-stable-rt.git
branch: v5.10-rt-next
commit: f1cf071b84981aa2e86d7664fbe7392956adb155
Maestro:
Build count:21
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-stable-rt.git
branch: v5.10-rt-next
commit: f1cf071b84981aa2e86d7664fbe7392956adb155
Dashboard:
Build count:21
Build count matched
URL: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
branch: master
commit: 35ff4c02b3d1c37df1090d011905e51390252064
Maestro:
Build count:17
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
branch: master
commit: 35ff4c02b3d1c37df1090d011905e51390252064
Dashboard:
Build count:17
Build count matched
URL: https://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git
branch: next
commit: 187715cfd12932a528ff3a3952648e2b55381d4c
Maestro:
Build count:17
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git
branch: next
commit: 187715cfd12932a528ff3a3952648e2b55381d4c
Dashboard:
Build count:17
Build count matched

maybe it could be:

tree/branch: <maestro> <dashboard> <status>
mainline/master: 17 17 ✅
next/master: 21 20 ❌
- Missing build IDs:['684b0042dba9af5ad63098c2']
...

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch 2 times, most recently from 2c0f26b to 586322c Compare June 16, 2025 09:11
@JenySadadia
Copy link
Author

Thanks for the PR @JenySadadia. I found a few issues:

Thank you for testing the PR and feedback.

  1. It seems the code is trying to look for a local git tree even if I'm supplying --giturl:
poutine kci-dev > poetry run kci-dev validate build --giturl https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git --branch master --commit 7d4e49a77d9930c69751b9192448fda6ff9100f1 
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
branch: master
commit: 7d4e49a77d9930c69751b9192448fda6ff9100f1
Maestro:
Build count:45
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
branch: master
commit: 7d4e49a77d9930c69751b9192448fda6ff9100f1
Dashboard:
Build count:45
Build count matched

The patch doesn't look for local git repo when URL, branch, and commit are provided.
The git folder:None print is from the existing utility I am using for fetching local repo if the options are not provided.
See https://github.com/kernelci/kci-dev/blob/main/kcidev/libs/git_repo.py#L300
We can tweak that function if the prints are misleading.

  1. if no commands are provided and the git tree doesn't exist in the dashboard, we should fail much cleaner than this:
poutine kci-dev > poetry run kci-dev validate build
git folder: None
tree: https://github.com/kernelci/kci-dev.git
branch: validate-builds-cmd
commit: 6b883f19e50f5750f3aa8757b462f6008658b017
Maestro:
Build count:0
git folder: None
tree: https://github.com/kernelci/kci-dev.git
branch: validate-builds-cmd
commit: 6b883f19e50f5750f3aa8757b462f6008658b017
json error: No results available for this tree/branch/commit
Aborted while fetching dashboard builds
Build count matched
  1. all checkouts out too verbose:
URL: https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-stable-rt.git
branch: v5.10-rt-next
commit: f1cf071b84981aa2e86d7664fbe7392956adb155
Maestro:
Build count:21
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-stable-rt.git
branch: v5.10-rt-next
commit: f1cf071b84981aa2e86d7664fbe7392956adb155
Dashboard:
Build count:21
Build count matched
URL: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
branch: master
commit: 35ff4c02b3d1c37df1090d011905e51390252064
Maestro:
Build count:17
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
branch: master
commit: 35ff4c02b3d1c37df1090d011905e51390252064
Dashboard:
Build count:17
Build count matched
URL: https://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git
branch: next
commit: 187715cfd12932a528ff3a3952648e2b55381d4c
Maestro:
Build count:17
git folder: None
tree: https://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git
branch: next
commit: 187715cfd12932a528ff3a3952648e2b55381d4c
Dashboard:
Build count:17
Build count matched

maybe it could be:

tree/branch: <maestro> <dashboard> <status>
mainline/master: 17 17 ✅
next/master: 21 20 ❌
- Missing build IDs:['684b0042dba9af5ad63098c2']
...

Thanks for the suggestion. Yes, I'll work on reformatting the output once I stabilize the command's results.
While investigating missing builds from this command output, I see that I need to use multiple filters with node.state to get the correct output.
I need to enable that support in the maestro API first.
Once I am done with that, we'll work on formatting the output.

@padovan
Copy link
Contributor

padovan commented Jun 16, 2025

The git folder:None print is from the existing utility I am using for fetching local repo if the options are not provided.
See https://github.com/kernelci/kci-dev/blob/main/kcidev/libs/git_repo.py#L300
We can tweak that function if the prints are misleading.

Yes, this should be fixed too.

@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch 2 times, most recently from 9383dbd to 14d1afd Compare June 17, 2025 06:50
@JenySadadia
Copy link
Author

@padovan I have reformatted the output.
How does this look?

┌──────────────────────────┬────────────────────────────────┬─────────┬───────────┬─────────────┬──────────────────────────────┬─────────────────┐
│ tree/branch              │ Commit                         │ Maestro │ Dashboard │ Build count │ Missing build IDs            │ Builds with     │
│                          │                                │ builds  │ builds    │ comparison  │                              │ status mismatch │
├──────────────────────────┼────────────────────────────────┼─────────┼───────────┼─────────────┼──────────────────────────────┼─────────────────┤
│ android/android13-5.15   │ 06f12cdca15807c094c59de63e5229 │ 24      │ 24        │ ✅          │ []                           │ []              │
│                          │ 74b3568f6c                     │         │           │             │                              │                 │
├──────────────────────────┼────────────────────────────────┼─────────┼───────────┼─────────────┼──────────────────────────────┼─────────────────┤
│ android/android-mainline │ 6985e40c1a0742a9f6c399b382a94f │ 34      │ 32        │ ❌          │ ['6851e4795c2cf25042bb2a5c', │ []              │
│                          │ 4dd1a1100a                     │         │           │             │ '6851e4a85c2cf25042bb2a89']  │                 │
├──────────────────────────┼────────────────────────────────┼─────────┼───────────┼─────────────┼──────────────────────────────┼─────────────────┤
│ android/android16-6.12   │ 0697e85aef2e6a986214cc886be423 │ 34      │ 34        │ ✅          │ []                           │ []              │
│                          │ fee63ae0e2                     │         │           │             │                              │                 │
└──────────────────────────┴────────────────────────────────┴─────────┴───────────┴─────────────┴──────────────────────────────┴─────────────────┘

Jeny Sadadia added 3 commits June 18, 2025 10:03
For the command `kci-dev results trees`, add `--days`
option to provide a period of time in days to
get results for.

Signed-off-by: Jeny Sadadia <[email protected]>
Make `kci_msg_*` functions more generic in terms of
new line at the end of the message.
Add a parameter to specify new line requirement while
printing message on terminal.
Update all related existing function calls.

Signed-off-by: Jeny Sadadia <[email protected]>
Replace `kci_msg` prints in `set_giturl_branch_commit`
with `logging.info` to avoid having too verbose output
on the terminal.

Signed-off-by: Jeny Sadadia <[email protected]>
@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch 5 times, most recently from 8c1d86b to abfc37b Compare June 18, 2025 09:21
@padovan
Copy link
Contributor

padovan commented Jun 18, 2025

It looks good to me! Thanks for reworking it!

@JenySadadia
Copy link
Author

It looks good to me! Thanks for reworking it!

Okay. Thanks :)

@JenySadadia
Copy link
Author

@aliceinwire Could you please review the PR?

Get builds from dashboard and validate
them with maestro result.

Command introduced with the commit:
`kci-dev validate build --all-checkouts --giturl <URL> \
--commit <commit-hash> --branch <git-branch>`
Provide `--all-checkouts` to build validation stats for all
available checkouts.
Build validation for a specific checkout can be performed by
using all three options: --giturl, --branch, and --commit.

The command will provide below information:
1. Build count validation
2. Missing build information
3. Build status validation

Used python package `tabulate` to generate
build stats report in tabular format.

Signed-off-by: Jeny Sadadia <[email protected]>
@JenySadadia JenySadadia force-pushed the validate-builds-cmd branch from abfc37b to 96af022 Compare June 19, 2025 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add kci-dev validate-builds to validate builds maestro <-> dashboard
3 participants