Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
kojix2 committed Apr 17, 2024
1 parent 687d417 commit e128463
Showing 1 changed file with 39 additions and 28 deletions.
67 changes: 39 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,63 @@
# Red DataStock (tentative name)
# Red DataStock

Forked from https://github.com/red-data-tools/red-datasets
## Overview

## Description
Red DataStock is a Ruby library designed to manage data download, cache, and extraction from specific URLs.
Useful when you need to consistently retrieve, cache, and process data files from the Internet.

Red DataStock provides a library for downloading and caching data.
## Installation

## Install
To install Red DataStock:

```
gem install red-datastock
```

## Usage

To use Red DataStock in your code, you need to require it first:

```ruby
module YourModule
class YourClass
# some nice code here...
```rb
require 'datastock'
```

Now you can use the `Downloader`, `CachePath`, and `ZipExtractor` classes for managing your data.

Here's an example of how to download, cache, and extract a zip file using Red DataStock:

```rb
require 'datastock'

def clear_cache!
cache_path.remove
end
# Prepare the Downloader with your desired URL
downloader = DataStock::Downloader.new("http://example.com/data.zip")

private
# Define where you want to cache the downloaded data
cache_path = DataStock::CachePath.new("my_data_id")

def cache_dir_path
cache_path.base_dir
end
# Define your output path
output_path = cache_path.base_dir + "data.zip"

def cache_path
@cache_path ||= CachePath.new(@metadata.id)
end
# Download and cache the data if it's not already cached
downloader.download(cache_path) unless File.exist?(output_path)

def download(output_path, url)
downloader = Downloader.new(url)
downloader.download(output_path)
end
end
# Prepare the ZipExtractor with your cached data
zip_extractor = DataStock::ZipExtractor.new(output_path)

# Extract files from the zip
zip_extractor.extract_files do |input|
# Your code to process each input file
end

# Clean up the cache directory
cache_path.clean
```

## Development
In this example, we download a zip file from `http://example.com/data.zip`, store it in a cache directory, and then extract all files from the zip. You can replace the URL with your actual data source.

Pull requests are welcome.
## Contributing

## License
Please create a fork, make your proposed changes, and submit a pull request.

The MIT license.
## License

The MIT License

0 comments on commit e128463

Please sign in to comment.