Iterate binds in a more efficient manner

[Frontear]

I'm using bwrap to write essentially a form of containerization where configurations are loaded into a bwrap via use of --binds. In order to ensure ease of use some user configuration is automatically also --bind-ed into the bwrap. This can essentially be visualized like this:

bwrap --bind /user-cfg /cfg --bind /load-cfg /cfg

where load-cfg are supposed to stack on top of the cfg.

The first issue was that /load-cfg was shadowing the previous bind (no /user-cfg). The solution to this was to iteratively bind each inner directory of /load-cfg on top of the fully binded /user-cfg. This resulted in the next problem: propagation into the /user-cfg. Neither of these are acceptable situations, so the final solution ended up building the configuration by processing all the inners:

for e in /user-cfg/*; do
  args+=(--bind "$e" "/cfg/...")
done

for e in /load-cfg/*; do
  args+=(--bind "$e" "/cfg/...")
done

This can result in very long startup times if there are many directories (the current amount we are hitting is about 200). Are there other ways that can be considered? We thought of using a second script within the bwrap to overlayfs the configuration directory, but we want changes done during the bwrap to propagate into the /user-cfg, which would be lost in a overlayfs.

Any suggestions? Looking through both the manual and the --help doesn't glean any helpful flag.

[rusty-snake]

Duplicated of #384

---

--bind /user-cfg /cfg/.user-cfg
--bind /load-cfg /cfg/.load-cfg
--symlink .user-cfg/foo /cfg/foo
--symlink .user-cfg/bar /cfg/bar
--symlink .load-cfg/baz /cfg/baz

[Frontear]

Symlinks arent feasible for directories, or dirs in dirs, and needing to iteratively symlink every single file would be have too many args and exhaust the stack (this has happened with bwrap complaining "too many args")

[rusty-snake]

You can avoid the "too many args" by pipeing the arguments with --fd.

There is no unprivileged unionfs/mergefs (if they meat your criteria) in the kernel. Maybe a userspace FUSE implementation (not in the scope of bwrap).

[smcv]

I think you want an overlay filesystem, #412/#547?

[Frontear]

Hmm this is something to try then. Thank you!

[Frontear]

I think you want an overlay filesystem, #412/#547?

EDIT: In my haste I did not realize the second link is a PR. That would actually be the most ideal solution, and adding extra work to copy changes from the work-dir could be a good alternative to what's happening right now.

Overlay was actually another thing we considered, but the problem was it wouldn't propagate the data downwards into /user-cfg, at least not without some extra work (manual/adding a script to cp -R work-dir ...).

It would actually be really nice, but since it's not part of bwrap at the moment, it cant be used unfortunately.

[smcv]

Oh, if an overlayfs doesn't meet your requirements, then there is probably nothing that bubblewrap can do for you. Writing a FUSE filesystem to combine trees in a different way is outside the scope of this project.

[Frontear]

No worries, this is quite a convoluted requirement. An overlayfs could be a pretty good solution, there would simply need to be some extra work to merge the trees, something that I could easily add for my own use-case.

I will implement --args FD and --symlink and hopefully this can be an ideal solution. If not I am eager to see the overlayfs PR make it through. Thanks for the information!