Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to view files in the archive #192

Closed
adamantida opened this issue Feb 6, 2024 · 8 comments
Closed

Ability to view files in the archive #192

adamantida opened this issue Feb 6, 2024 · 8 comments
Labels
enhancement New feature or request fixready
Milestone

Comments

@adamantida
Copy link

Hello, I have questions about viewing files in archives using the dwarfs program. In zpaqfranz there is a command "l" for this, but in dwarfs there is no such command (at least it is not mentioned anywhere). However, when using dwarfsck -d 4 it appears to be able to view files, but the results look strange. Are there plans to add a special command to view archive files in dwarfs in the future? Also wondering if there will be support for archive versions similar to zpaqfranz in future dwarfs updates?

@mhx
Copy link
Owner

mhx commented Feb 7, 2024

Hi and thanks for your feedback!

I guess that because DwarFS is primarily a file system rather than an archive format (although it seems that it's quite often used as such), this request has never really come up before. It'd be trivial to implement and I'll probably just do it if it's considered useful.

However, there is a way to already do this, but it's a bit of a hack:

$ dwarfsextract -i min.dwarfs -f mtree
I 08:00:56.986420 extraction finished without errors
#mtree
./bench.sh time=1616939327.0 mode=644 gid=100 uid=1000 type=file size=33
./foobar time=1606400164.0 mode=777 gid=100 uid=1000 type=link link=foo/bar
./format.sh time=1606256045.0 mode=755 gid=100 uid=1000 type=file size=94
./perl-exec.sh time=1606161908.0 mode=644 gid=100 uid=1000 type=file size=87
./dev time=1606845311.0 mode=755 gid=100 uid=1000 type=dir
./empty time=1606945040.0 mode=755 gid=100 uid=1000 type=dir
./empty/alsoempty time=1606945040.0 mode=755 gid=100 uid=1000 type=dir
./foo time=1616013726.0 mode=755 gid=100 uid=1000 type=dir
./foo/bad time=1606814322.0 mode=777 gid=100 uid=1000 type=link link=../foo
./foo/bar time=1606310580.0 mode=644 gid=100 uid=1000 type=file size=0
./foo/bla.sh time=1616939377.0 mode=644 gid=100 uid=1000 type=file size=33
./foo/1 time=1616013804.0 mode=755 gid=100 uid=1000 type=dir
./foo/1/fmt.sh time=1606256045.0 mode=755 gid=100 uid=1000 type=file size=94
./foo/1/2 time=1616013816.0 mode=755 gid=100 uid=1000 type=dir
./foo/1/2/xxx.sh time=1606256045.0 mode=755 gid=100 uid=1000 type=file size=94
./foo/1/2/3 time=1616939417.0 mode=755 gid=100 uid=1000 type=dir
./foo/1/2/3/copy.sh time=1616013831.0 mode=755 gid=100 uid=1000 type=file size=94

This uses libarchive's mtree format as output, which will simply list all files. While this works, the overhead is massive, as it'll decompress all the data even though none of it is used. The fastest way to list all files currently is to just mount the image.

The output of dwarfsck is really meant for debugging and seeing what's going on in the file system metadata. It's not meant for casual consumption. :)

Also wondering if there will be support for archive versions similar to zpaqfranz in future dwarfs updates?

Yes, but there's no firm timeline. This has been a request for a long time already, see also #18.

@mhx
Copy link
Owner

mhx commented Feb 7, 2024

@adamantida, the next release will have a -l / --list option for dwarfsck:

$ dwarfsck ../test/data.dwarfs -l 
foobar
format.sh
perl-exec.sh
test.py
unicode
unicode/我爱你
unicode/我爱你/☀️ Sun
unicode/我爱你/☀️ Sun/Γειά σας
unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا
unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️
unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️/Карибського
$ dwarfsck ../test/data.dwarfs -lv
---lrwxrwxrwx 1000/1000    7 2020-11-26 15:16 foobar -> foo/bar
----rwxr-xr-x 1000/1000   94 2020-11-24 23:14 format.sh
----rw-r--r-- 1000/1000   87 2020-11-23 21:05 perl-exec.sh
----rw-r--r-- 1000/1000 1012 2020-12-05 22:46 test.py
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你/☀️ Sun
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你/☀️ Sun/Γειά σας
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا
---drwxr-xr-x 1000/1000    1 2023-06-28 10:29 unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️
----rw-r--r-- 1000/1000    8 2023-06-28 10:29 unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️/Карибського

@mhx mhx added enhancement New feature or request fixready labels Feb 7, 2024
@mhx mhx added this to the v0.9.2 milestone Feb 7, 2024
@adamantida
Copy link
Author

adamantida commented Feb 7, 2024

Perhaps it will be useful. I want to say about the inconvenience of using --tool=dwarfsck in "universal" is too long and inconvenient, instead it is more convenient, in my opinion, to use short designations in the form of letters without "-", let's say there is dwarfsextract, the name speaks for itself, it is convenient to use as "./dwarfs x -i min.dwarfs -f mtree", as well as "./dwarfs x min.dwarfs -f mtree". This is more convenient and easier to use.
I also noticed that when using universal dwarfs on windows --tool=dwarfs - an error pops up, which is natural without the .dll
But in normal bin for using ./dwarfs it doesn't pop up.

@mhx
Copy link
Owner

mhx commented Feb 7, 2024

Perhaps it will be useful. I want to say about the inconvenience of using --tool=dwarfsck in "universal" is too long and inconvenient, instead it is more convenient, in my opinion, to use short designations in the form of letters without "-", let's say there is dwarfsextract, the name speaks for itself, it is convenient to use as "./dwarfs x -i min.dwarfs -f mtree", as well as "./dwarfs x min.dwarfs -f mtree". This is more convenient and easier to use.

I'm not a big fan of these single letter commands. I agree that dwarfs-universal --tool=dwarfsck is verbose, but then again, you have the choice of using symlinks, you can just create copies of the universal binary with different names, or you can install the regular binaries.

I also noticed that when using universal dwarfs on windows --tool=dwarfs - an error pops up, which is natural without the .dll But in normal bin for using ./dwarfs it doesn't pop up.

Can you elaborate?

Without the WinFsp DLL in the PATH, running

> dwarfs-universal --tool=dwarfs

will show a message in the terminal:

failed to load winfsp-x64.dll

Whereas running

> dwarfs

will show a pop-up dialog saying that winfsp-x64.dll could not be loaded.

Once I put the WinFsp DLL in the PATH, both tools work just fine.

@adamantida
Copy link
Author

Can you elaborate?

Yeah, sure

image

image
The dialog box itself is missing. The error is not displayed that winfsp-x64.dll could not be loaded

Once I put the WinFsp DLL in the PATH, both tools work just fine.

I understand that perfectly well, but I'm talking about the fact that the error window itself doesn't appear exactly when using ".\dwarfs.exe"

@mhx
Copy link
Owner

mhx commented Feb 7, 2024

Looks like it's a PowerShell specific issue...
Screenshot 2024-02-07 at 23 29 01

@mhx
Copy link
Owner

mhx commented Feb 7, 2024

Indeed: PowerShell/PowerShell#16468

@mhx
Copy link
Owner

mhx commented Feb 8, 2024

I've added a workaround for the power shell issue by using delayed loading also for dwarfs.exe; that will avoid the pop-up and rather show the same error message as when using the universal binary. This will be part of the next release.

@mhx mhx closed this as completed Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request fixready
Projects
None yet
Development

No branches or pull requests

2 participants