Can't mount more filesystems within a read-only mount

[pxeger]

Simple example:

$ bwrap --ro-bind / / --bind . /here true
bwrap: Can't mkdir /here: Read-only file system

This can be worked around by creating the target directory for the second mount inside the root of the first mount, or by using something like overlayfs.

If there was a way to work around this in bubblewrap itself, it would be nice, but I'm not sure it's possible. Maybe if overlayfs integration gets added (#412), there could be an option for a "semi-read-only" mount to avoid this issue.

Also, maybe there could be some sort of warning with hints to resolve it if this situation is detected?

While there may not be any practical solution, I'm partly also posting this so anyone else who has this issue doesn't need to spend their weekend debugging it.

[haampie]

Note that overlayfs is not required for this, since bwrap creates a new root in which you have write permissions, so you just have to avoid --bind'ing it with a source you don't have write access to.

So, if you instead mount children of /, everything is fine

example.sh

#!/bin/bash
args=()
shopt -s dotglob
for d in /*; do
  args+=("--dev-bind" "$d" "$d")
done
bwrap "${args[@]}" "$@"

$ ./example.sh --bind $(mktemp -d) /test stat /test
  File: /test
  Size: 4096      	Blocks: 8          IO Block: 4096   directory

An in general you can do ./example.sh mkdir /hello; this /hello directory is created in bubblewrap's (hidden) tmpfs and is gone after exiting the process.

It would be nice though if we don't have to use a wrapper script for it. And also using many --bind's is slow (#384)

[kit-ty-kate]

Sure, this works for directories, however it stops working if files are present in /.

e.g.

$ ./example.sh ls /
bwrap: Can't find source path /initrd.img: No such file or directory

[haampie]

It works for files too

[kit-ty-kate]

Can you share the corresponding shell script allowing files in /?

[haampie]

It's the same...

$ cat /file 
hello
$ bwrap --version 
bubblewrap 0.4.0
$ ./example.sh cat /file 
hello

Is there anything special about your root-level file?

[farblos]

@haampie: What if the file or directory B to inject into directory A is several layers below A?

More concretely, I'd like to have directory /usr ro-bound from my host and install and use an additional CA certificate in my bubblewrap container. On Debian, that must be a file below /usr/share/ca-certificates.

With the above approach, I'd need to bind mount /usr/* (except /usr/share), then /usr/share/* (except /usr/share/ca-certificates), etc.

Or have I missed something here?

[farblos]

Or have I missed something here?

Yep, at least in this particular case: Debian's update-ca-certificates has an option to specify the path of its "input directory", and this has not to be /usr/share/ca-certificates. Doh.

But in general it would be nice to have some tool to blend one directory into another without a lot of fuss ... fuse-overlayfs did not seem to work in a bubble-wrapped container, but maybe I have overlooked something there, too.

[smcv]

In general, bubblewrap is a low-level tool which tries to do what you ask for, even if what you ask for can't work. We don't have the opportunity to change the kernel's behaviour, and the kernel won't allow mounting something if there is no mount point, so all we can do is try to create mount points.

So, if you instead mount children of /, everything is fine

Yes (and this is how Flatpak implements --filesystem=host) - as long as you don't expect that creating a directory /new outside the sandbox will make /new appear inside the sandbox.

Maybe if overlayfs integration gets added (#412), there could be an option for a "semi-read-only" mount to avoid this issue.

I think #412 will probably already make this possible, without needing new options: mount the read-only directory (/ in your example) with a read/write tmpfs overlay on top, then mount the subdirectory /here, and finally --remount-ro / to switch the tmpfs overlay to be read-only.

maybe there could be some sort of warning with hints to resolve it if this situation is detected?

Merge requests welcome, although I don't think it will necessarily be straightforward.

Please bear in mind that bwrap is setuid root on some systems, so anything that is added to bwrap needs to be hardened against malicious users crafting a command-line to try to elevate privileges to root, which limits the dependencies and complexity that can be accepted. It's often better to put the mechanism in bwrap (make things possible, but not necessarily easy), and have a higher layer like Flatpak that is the policy (take a "do what I mean" high-level goal and turn it into potentially many bwrap command-line options). Flatpak's common/flatpak-exports.c is an example of the sort of thing that's necessary in practice.

[xi]

Wouldn't the --remout-ro option solve this?

bwrap --bind / / --bind . /here true --remount-ro /

The only downside I see is that the mountpoints that bubblewrap creates are still there afterwards.

[smcv]

@xi: If the mount point /here being created in (in your example) the real root is acceptable for your use-case, then that's an entirely valid mechanism to use to achieve your desired policy.

Again, bubblewrap is mechanism more than policy, so it can't know whether that's acceptable or not; implementing your desired policy is a job for a higher-layer thing like Flatpak or your d-utils.