Support for gz zip files (feature request)

[jhelvy]

Hi! Loving this packing!

I wanted to ask if support for gz files could be added. Right now if I use the zip_to_disk.frame() function on a .gz file I get an error. Here's a simple example:

> library(disk.frame)
> write.csv(mtcars, "mtcars.csv") # Create csv
> system("gzip mtcars.csv") # Zip it to .gz file
> df <- zip_to_disk.frame("mtcars.csv.gz") # Attempt to read .gz file to disk.frame

This produces this error:

Error in unzip(zipfile, list = TRUE) : 
  zip file 'mtcars.csv.gz' cannot be opened

My solution for now is to use {vroom} to read in the .gz and then convert it to a disk.frame:

library(vroom)
df <- as.disk.frame(vroom("mtcars.csv.gz"))

It works just fine, but I have a file with several hundred .gz files so it'd be nicer if I could just point to the folder and have {disk.frame} read them in sequentially.

[xiaodaigh]

I see. For now I think this would work

list_of_disk.frames = lapply(files, function(file) {
  as.disk.frame(vroom(file))
})


fnl_disk.frame = rbindlist.disk.frame(list_of_disk.frames)

[jhelvy]

Oh nice work around! I didn't even know about rbindlist.disk.frame().
This should work as a generic solution too for lots of different file types.

[xiaodaigh]

Yep. The only think I would say is that you can parallelize the lapply using future.apply