Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse tar data backed up via stdin #1381

Open
dionorgua opened this issue Dec 8, 2024 · 1 comment
Open

Parse tar data backed up via stdin #1381

dionorgua opened this issue Dec 8, 2024 · 1 comment
Labels
A-core Area: Generally related to `rustic_core` C-enhancement Category: New feature or request

Comments

@dionorgua
Copy link

dionorgua commented Dec 8, 2024

Hi,

It would be cool to have a way to 'import' tar archive as stdin backup.

Something like

tar c <....> | rustic backup -

Why? Sometimes it's too hard to get access to actual data that needs to be saved.
For example proxmox have vzdump command that can save backups to .tar.gz/zstd format on disk or can output tar stream to stdout. Surely we can 'unpack' that tar temporary but this extra I/O that can be avoided

Same as restic/restic#2226

@github-actions github-actions bot added the S-triage Status: Waiting for a maintainer to triage this issue/PR label Dec 8, 2024
@aawsome
Copy link
Member

aawsome commented Jan 19, 2025

Thanks for opening this issue @dionorgua!

I also want to add this feature but I must say that some more groundwork is needed. I'll describe a bit the problem:

  • rustic backup is optimized to read files in parallel (which is an already implemented performance tuning)
  • To allow this, the scanning of files (i.e. get the information which files are to backup) and the actual reading is separated and may occur in multiple thread
  • This is incompatible with reading tar data from a stream, as here we get the information about the contained file directly followed by the content before we get the information of the next file.

Besides solving this problem we also have to enhance the possibility to config backup sources, e.g. how to differentiate whether to interpret the stdin stream as tar or just as content. But this is the smaller part of the problem...

@aawsome aawsome added C-enhancement Category: New feature or request A-core Area: Generally related to `rustic_core` and removed S-triage Status: Waiting for a maintainer to triage this issue/PR labels Jan 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-core Area: Generally related to `rustic_core` C-enhancement Category: New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants