-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
39 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,52 +1,63 @@ | ||
# Red DataStock (tentative name) | ||
# Red DataStock | ||
|
||
Forked from https://github.com/red-data-tools/red-datasets | ||
## Overview | ||
|
||
## Description | ||
Red DataStock is a Ruby library designed to manage data download, cache, and extraction from specific URLs. | ||
Useful when you need to consistently retrieve, cache, and process data files from the Internet. | ||
|
||
Red DataStock provides a library for downloading and caching data. | ||
## Installation | ||
|
||
## Install | ||
To install Red DataStock: | ||
|
||
``` | ||
gem install red-datastock | ||
``` | ||
|
||
## Usage | ||
|
||
To use Red DataStock in your code, you need to require it first: | ||
|
||
```ruby | ||
module YourModule | ||
class YourClass | ||
# some nice code here... | ||
```rb | ||
require 'datastock' | ||
``` | ||
|
||
Now you can use the `Downloader`, `CachePath`, and `ZipExtractor` classes for managing your data. | ||
|
||
Here's an example of how to download, cache, and extract a zip file using Red DataStock: | ||
|
||
```rb | ||
require 'datastock' | ||
|
||
def clear_cache! | ||
cache_path.remove | ||
end | ||
# Prepare the Downloader with your desired URL | ||
downloader = DataStock::Downloader.new("http://example.com/data.zip") | ||
|
||
private | ||
# Define where you want to cache the downloaded data | ||
cache_path = DataStock::CachePath.new("my_data_id") | ||
|
||
def cache_dir_path | ||
cache_path.base_dir | ||
end | ||
# Define your output path | ||
output_path = cache_path.base_dir + "data.zip" | ||
|
||
def cache_path | ||
@cache_path ||= CachePath.new(@metadata.id) | ||
end | ||
# Download and cache the data if it's not already cached | ||
downloader.download(cache_path) unless File.exist?(output_path) | ||
|
||
def download(output_path, url) | ||
downloader = Downloader.new(url) | ||
downloader.download(output_path) | ||
end | ||
end | ||
# Prepare the ZipExtractor with your cached data | ||
zip_extractor = DataStock::ZipExtractor.new(output_path) | ||
|
||
# Extract files from the zip | ||
zip_extractor.extract_files do |input| | ||
# Your code to process each input file | ||
end | ||
|
||
# Clean up the cache directory | ||
cache_path.clean | ||
``` | ||
|
||
## Development | ||
In this example, we download a zip file from `http://example.com/data.zip`, store it in a cache directory, and then extract all files from the zip. You can replace the URL with your actual data source. | ||
|
||
Pull requests are welcome. | ||
## Contributing | ||
|
||
## License | ||
Please create a fork, make your proposed changes, and submit a pull request. | ||
|
||
The MIT license. | ||
## License | ||
|
||
The MIT License |