Task fetch via zip lost perms, this cause Restraint to kill given task

[StykMartin]

I executed Restraint with one task. The task should be fetched and content is stored in a zip file.

This is what Restraint unpacked in given system

[root@slave-04 mnt]# ls -la tests/github.com/beaker-project/beaker-core-tasks/archive/master.zip/reservesys/
total 24
drwxr-xr-x. 2 root root   77 Jul 29 08:10 .
drwxr-xr-x. 3 root root   24 Jul 29 08:10 ..
-rw-r--r--. 1 root root 1545 Jul 29 08:10 Makefile
-rw-r--r--. 1 root root  156 Jul 29 08:10 metadata
-rw-r--r--. 1 root root 1463 Jul 29 08:10 recipe_status
-rw-r--r--. 1 root root 9112 Jul 29 08:10 runtest.sh

I made curl to the same source to check if we screw up data

[root@slave-04 mnt]# ls -la beaker-core-tasks-master/reservesys/
total 24
drwxrwxr-x.  2 root root   77 Jul 12 05:08 .
drwxrwxr-x. 12 root root  287 Jul 12 05:08 ..
-rw-rw-r--.  1 root root 1545 Jul 12 05:08 Makefile
-rw-rw-r--.  1 root root  156 Jul 12 05:08 metadata
-rwxrwxr-x.  1 root root 1463 Jul 12 05:08 recipe_status
-rwxrwxr-x.  1 root root 9112 Jul 12 05:08 runtest.sh

All looks good. So it seems that our libarchive can't handle zip format properly.
I extended our unit tests and this is not happening with tar balls. You can check #135.

[StykMartin]

I was able to easily reproduce given behavior

SoupURI *url = soup_uri_new("https://github.com/beaker-project/beaker-core-tasks/archive/master.zip#reservesys");

Then when I check temp directory created in unit test

➜   ls -la /tmp/test_fetch_http_UGLVO0 
total 24
drwxrwxr-x.   2 mastyk mastyk  120 Jul 30 09:29 .
drwxrwxrwt. 237 root   root   4860 Jul 30 09:29 ..
-rw-rw-r--.   1 mastyk mastyk 1545 Jul 30 09:29 Makefile
-rw-rw-r--.   1 mastyk mastyk  156 Jul 30 09:29 metadata
-rw-rw-r--.   1 mastyk mastyk 1463 Jul 30 09:29 recipe_status
-rw-rw-r--.   1 mastyk mastyk 9112 Jul 30 09:29 runtest.sh

[ernestask]

Dumping my findings here:

libarchive chooses the streaming decompressor whenever there is no seek callback defined (which is the case whenever not using archive_read_open_fd(), archive_read_open_FILE() or any other friend). As the name would imply, it is incapable of seeking to the central directory and reading the complete file attributes, so you will not get the original permissions (c.f. libarchive/libarchive#1106).

So what do now?

Since the compressed data is backed by GMemoryInputStream, which implements GSeekable, one could easily use that to make libarchive select the seeking decompressor (note that restraint will first fetch all the data regardless, despite the process being asynchronous). Soon enough, however, you will realize that libarchive loves seeking past the end of the buffer, which would be unsupported by this particular memory stream (https://gitlab.gnome.org/GNOME/glib/-/issues/587).

In the end, libarchive would have to be changed to not do random seeks, GLib would have to be changed to be more like lseek() in that regard, or you would use a different data store that can handle seeking better (writing to a file and pointing libarchive to it is one such solution… memfd?).