Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report more informative error to user when same job is run concurrently with docker cache mounted #41

Open
ScottG489 opened this issue Jan 3, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@ScottG489
Copy link
Owner

Currently when two concurrent jobs try to access /var/lib/docker the second job will fail with errors only reported to the dockerd logs.

After a little more research, two jobs **can actually mount /var/lib/docker at the same time. However, it seems like the second job to run dockerd will result in dockerd failing to start because only one instance of the daemon can run at the same time.

This actually relaxes the requirement a bit in that only one instance of a job can run at the same time with /var/lib/docker mounted and only requires that one of them run dockerd.

However, this also makes reporting this error to the user more difficult since the problem occurs inside their job rather than at the outer docker level where we can monitor.

It seems like we'd have to somehow introspect the job to somehow see if dockerd was running at the same time or two things were trying to access the mounted /var/lib/docker. This seems rather invasive though.

A better alternative could be to warn the user if the job is being run concurrently and any attempts to run dockerd could result in errors.

This may even open the possibility to only running dockerd in jobs just in time for needing to run docker commands and then stopping the daemon afterwards. This would mitigate any overlap, but would not completely outrule failures.

Documentation of this limitation should also be made.

Finally, eventually we may want to add job configuration or query parameters to enable/disable mounting of /var/lib/docker and maybe of use of sysbox altogether if the user doesn't need this functionality. This way they would have to be explicit and more aware of limitations.

This is a follow up to #17.

@ScottG489 ScottG489 added the enhancement New feature or request label Jan 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant