syz-cluster: initial code #5620

a-nogikh · 2024-12-17T15:57:28Z

The basic code of a k8s-based cluster that:

Aggregates new LKML patch series.
Determines the kernel trees to apply them to.
Builds the basic and the patched kernel.
Displays the results on a web dashboard.

This is a very rudimentary version with a lot of TODOs that provides a skeleton for further work.

The project makes use of Argo workflows and Spanner DB.
Bootstrap is used for the web interface.

Overall structure:

syz-cluster/dashboard: a web dashboard listing patch series and their test results.
syz-cluster/series-tracker: polls Lore archives and submits the new patch series to the DB.
syz-cluster/controller: schedules workflows and provides API for them.
syz-cluster/kernel-disk: a cron job that keeps a kernel checkout up to date.
syz-cluster/workflow/*: workflow steps.

For the DB structure see syz-cluster/pkg/db/migrations/*.

TODO (for this PR):

Go through the installation section, hopefully get rid of minio use.
Figure out why go.mod bumped to 1.23.1
Figure out how to run syz-cluster tests on GitHub CI (we'll need at least some local Spanner emulator binary).

Questions/thoughts out loud:
1. What to do with the vendor folder
Our vendor/ folder is quite big, and this PR adds even more modules on top of that. Do we still want to keep that code in our repostory? Is it possible to only keep the modules needed by other components but not syz-cluster (since I do go mod download in the Docker containers anyway).
Filed #5645

2. Some pkg/ packages are too eager to depend on prog/ and sys/
It has made some Dockerfiles more complicated that they could have been.
Should we strive to remove these dependencies?

dvyukov

The first patch is not questionable. Let's review/merge it separately.

dvyukov · 2024-12-18T11:56:16Z

pkg/email/lore/parse.go

-var patchSubjectRe = regexp.MustCompile(`\[(?:(?:rfc|resend)\s+)*patch`)
+type PatchSubject struct {
+	Title   string
+	Tags    []string // Sometimes there's e.g. "net" or "next-next" in the subject.


These tags are parsed, but lost later. What's the plan here?
Do we want to not test "RFC" patches? Then these should set Corrupted?

As mentioned in the comment, the developers sometimes explicitly say that the the patch it to be applied to the specific tree. This information is not used in this initial PR, but will be important for the further series triage implementations.

pkg/email/lore/parse.go

dvyukov · 2024-12-18T11:58:53Z

pkg/email/lore/parse.go

@@ -20,9 +22,29 @@ type Thread struct {
 	Messages  []*email.Email
 }

+type Series struct {
+	Subject   string


Does it happen that the subject changes between versions?
If so, do we need to also extract in-reply to message-id, which may be the previous version?

AFAIK the series are usually not sent in-reply-to the previous ones, so I'd expect the title to be fairly stable. Otherwise there's no chance to relate them.

pkg/osutil/osutil.go

pkg/vcs/git.go

dvyukov · 2024-12-18T12:11:35Z

pkg/vcs/git.go

+	Dir      string
+	Precious bool
+	Sandbox  bool
+	Env      []string


Can we unexport some of these fields if they are used only within this package?

I've unexported Precious, the others are used outside (at least so far).

pkg/vcs/git.go

Support filtering by the commit date.

Refactor the code to make it more reusable. Add a method to extract specifically the list of new patch series.

It will let other parts of the system to use the git-specific functionality.

The basic code of a K8S-based cluster that: * Aggregates new LKML patch series. * Determines the kernel trees to apply them to. * Builds the basic and the patched kernel. * Displays the results on a web dashboard. This is a very rudimentary version with a lot of TODOs that provides a skeleton for further work. The project makes use of Argo workflows and Spanner DB. Bootstrap is used for the web interface. Overall structure: * syz-cluster/dashboard: a web dashboard listing patch series and their test results. * syz-cluster/series-tracker: polls Lore archives and submits the new patch series to the DB. * syz-cluster/controller: schedules workflows and provides API for them. * syz-cluster/kernel-disk: a cron job that keeps a kernel checkout up to date. * syz-cluster/workflow/*: workflow steps. For the DB structure see syz-cluster/pkg/db/migrations/*.

a-nogikh · 2024-12-26T10:30:42Z

FTR:

We could add a standalone Spanner emulator binary to our syz-env image like described here: https://github.com/GoogleCloudPlatform/cloud-spanner-emulator/blob/master/README.md#via-pre-built-linux-binaries
parallelism: 1 and failFast: true could be configured per workflow template: https://argo-workflows.readthedocs.io/en/latest/fields/#workflowspec. Thus the iterateItems step could be made its own template, but there's still a question on how to distinguish between "try the next kernel revision" and "I've managed to build this one. Stop iterating.".
Lifecycle hooks seem to be the recommended way of implementing onExit callbacks: https://argo-workflows.readthedocs.io/en/latest/lifecyclehook/
MinIO seems be unavoidable for local argo workflow runs, there are no simpler artifact storage options.

a-nogikh force-pushed the features/syz-cluster branch from dbf79ae to 0a8c2df Compare December 17, 2024 15:57

a-nogikh changed the title ~~Features/syz cluster~~ syz-cluster: initial code Dec 17, 2024

dvyukov reviewed Dec 18, 2024

View reviewed changes

pkg/osutil/osutil.go Outdated Show resolved Hide resolved

dvyukov reviewed Dec 18, 2024

View reviewed changes

a-nogikh force-pushed the features/syz-cluster branch from 0a8c2df to 3ef5a48 Compare December 19, 2024 21:36

a-nogikh mentioned this pull request Dec 19, 2024

pkg/osutil: add a WriteJSON helper #5631

Merged

pkg/vcs: extend ListCommitHashes

7f21017

Support filtering by the commit date.

a-nogikh force-pushed the features/syz-cluster branch from 3ef5a48 to 2a65760 Compare December 20, 2024 21:42

a-nogikh mentioned this pull request Dec 23, 2024

pkg/vcs: extend ListCommitHashes #5643

Open

pkg/email/lore: extract patch series

cbbdc61

Refactor the code to make it more reusable. Add a method to extract specifically the list of new patch series.

a-nogikh force-pushed the features/syz-cluster branch from 2a65760 to 770edae Compare December 23, 2024 11:37

pkg/vcs: export GitWrapper

259a2d8

It will let other parts of the system to use the git-specific functionality.

a-nogikh force-pushed the features/syz-cluster branch 2 times, most recently from ae3f62b to 0129842 Compare December 23, 2024 15:05

a-nogikh force-pushed the features/syz-cluster branch from 0129842 to 14daace Compare December 24, 2024 10:15

a-nogikh mentioned this pull request Dec 24, 2024

syzkaller: switch to -mod=mod? #5645

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

syz-cluster: initial code #5620

syz-cluster: initial code #5620

a-nogikh commented Dec 17, 2024 •

edited

Loading

dvyukov left a comment

dvyukov Dec 18, 2024

a-nogikh Dec 23, 2024

dvyukov Dec 18, 2024

a-nogikh Dec 23, 2024 •

edited

Loading

dvyukov Dec 18, 2024

a-nogikh Dec 23, 2024

a-nogikh commented Dec 26, 2024 •

edited

Loading

syz-cluster: initial code #5620

Are you sure you want to change the base?

syz-cluster: initial code #5620

Conversation

a-nogikh commented Dec 17, 2024 • edited Loading

dvyukov left a comment

Choose a reason for hiding this comment

dvyukov Dec 18, 2024

Choose a reason for hiding this comment

a-nogikh Dec 23, 2024

Choose a reason for hiding this comment

dvyukov Dec 18, 2024

Choose a reason for hiding this comment

a-nogikh Dec 23, 2024 • edited Loading

Choose a reason for hiding this comment

dvyukov Dec 18, 2024

Choose a reason for hiding this comment

a-nogikh Dec 23, 2024

Choose a reason for hiding this comment

a-nogikh commented Dec 26, 2024 • edited Loading

a-nogikh commented Dec 17, 2024 •

edited

Loading

a-nogikh Dec 23, 2024 •

edited

Loading

a-nogikh commented Dec 26, 2024 •

edited

Loading