Supporting fs.FS alongside a local directory in Solve's LocalDirs implementation?

[hugosantos]

We have a few use-cases where we precompute the filesystem to be used by a buildkit invocation, and would benefit from being able to have buildkit consume an arbitrary "filesystem", as opposed to only supporting diffcopy from the local fs.

We've considering creating individual files using llb.Mkfile etc but we could be creating more than 1000 files that are served as input, and don't always know which ones exactly will be used by invocation. So being able to more incrementally upload the contents would be beneficial.

If this makes sense, happy to send a pull request. Thanks!

[tonistiigi]

This should probably be ok as update to tonistiigi/fsutil without any buildkit API change. I'm not sure if you can use fs.FS though. I think you more likely need something like https://pkg.go.dev/github.com/tonistiigi/fsutil#FS

[hugosantos]

Using https://pkg.go.dev/github.com/tonistiigi/fsutil#FS SGTM.

Do you have some thoughts on how to accomplish this without changes to the buildkit API?

The options I've explored so far are preparing a Session, with a custom filesync implementation, but prepareSyncedDirs fails the call if llb.Local is used without a corresponding LocalDirs entry.

And the fsutil.FS instance is constructed by filesync.SyncedDir, so a constructor func would have to be passed in?

Another option would be to use the gRPC API directly and replicate solve(), but ideally we'd fork as little as possible -- ideally zero. :-)

[tonistiigi]

Actually, I don't think it needs fsutil changes. All these functions seem to already use FS.

So you need a way to pass FS from client so instead of NewFS(string) call in https://github.com/moby/buildkit/blob/master/session/filesync/filesync.go#L108 it would use it directly.