Skip to content

Commit

Permalink
support format without protocol
Browse files Browse the repository at this point in the history 
- update s3handler
- support format without protocol
  • Loading branch information
Antonio Yang committed Sep 22, 2019
1 parent b025d8f commit e1bf918
Show file tree
Hide file tree
Showing 3 changed files with 61 additions and 53 deletions.
11 changes: 3 additions & 8 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[package]
name = "s3rs"
version = "0.2.8"
version = "0.2.9"
authors = ["Antonio Yang <[email protected]>"]
description = "A s3 cli client with multi configs with diffent provider"
keywords = ["S3", "Amazon", "ceph", "AWS"]
keywords = ["S3", "Amazon", "CEPH", "AWS"]
categories = ["command-line-interface"]
license = "MIT"
readme = "README.md"
Expand Down Expand Up @@ -35,9 +35,4 @@ serde_json = "1.0"
regex = "0.2"
quick-xml = "0.12"
colored = "1.6"
s3handler = "0.3.2"

# Patch for s3handler
# [patch.crates-io]
# s3handler = { path = "../s3handler" }
# s3handler = { git = "https://github.com/yanganto/s3handler", branch = "master" }
s3handler = "0.3.3"
86 changes: 46 additions & 40 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,54 +11,59 @@ an **S3** **R**ich **S**upport client
- support http redirect for multi region of AWS S3
- support automatically multipart upload
- support filters [v0.2.8](https://www.ant-lab.tw/2019-09-21/)
- support format without protocol

# Download excutable binary
Download the link as follow and unzip
- https://github.com/yanganto/s3rs/releases/download/v0.2.8/s3rs-v0.2.8-linux.zip
- https://github.com/yanganto/s3rs/releases/download/v0.2.9/s3rs-v0.2.9-linux.zip

# Command List

| COMMAND | FUNCTION | CEPH | AWS |
|----------------------------------------------------------------------|------------------------------------------------|------|-----|
| ls | list all buckets | O | O |
| mb _bucket_ | create bucket | O | O |
| rb _bucket_ | delete bucket | O | O |
| put <file> s3://_bucket_/_object_ | upload the file | O | O |
| put <file> s3://_bucket_ | upload the file (file name as object name) | O | O |
| put test s3://_bucket_/_object_ | upload a test file sepcific object name | O | O |
| get s3://_bucket_/_object_ _file_ | download object | O | O |
| get s3://_bucket_/_object_ | download object in current folder | O | O |
| cat s3://_bucket_/_object_ | show the object content | O | O |
| del s3://_bucket_/_object_ [delete-marker:true] [secure-delete:true] | delete the object (with flag) | O | O |
| | delete-marker used in AWS | | |
| | secure-delete used in BIGTERA(customized CEPH) | | |
| tag list s3://_bucket_/_object_ | list tag(s) to the object | O | O |
| tag ls s3://_bucket_/_object_ | list tag(s) to the object | O | O |
| tag add s3://_bucket_/_object_ _key1_=_value1_ [_key2_=_value2_] ... | add tag(s) to the object | O | O |
| tag put s3://_bucket_/_object_ _key1_=_value1_ [_key2_=_value2_] ... | add tag(s) to the object | O | O |
| tag del s3://_bucket_/_object_ | remove tag(s) from the object | O | O |
| tag rm s3://_bucket_/_object_ | remove tag(s) from the object | O | O |
| /uri?query | give the orignal url | O | O |

| HIGH LEVEL COMMAND | INTEGRATE FUNCTIONS | CEPH | AWS |
|---------------------------|------------------------------------------------------------------------------------------|------|-----|
| la | list all objects | O | O |
| ls _bucket_ | list objects in the bucket | O | O |
| ls s3://_bucket_ | list objects in the bucket | O | O |
| ls s3://_bucket_/_prefix_ | list objects match prefix in the bucket | O | O |
| ll | list all objects details (storage class, modify time, etag) | O | O |
| ll s3://_bucket_ | list objects detail in the bucket | O | O |
| ll s3://_bucket_/_prefix_ | list objects match prefix detail in the bucket | O | O |
| info s3://_bucket_ | acl(ceph, aws), location(ceph, aws), versioning(ceph, aws), uploads(ceph), version(ceph) | O | O |

| CEPH OPS API (use system keys) | CEPH ONLY FUNCTION |
|--------------------------------|-----------------------|
| usage s3://_bucket_ | show the bucket usage |


#### Table 1: List commands will send single http request
| COMMAND | FUNCTION | CEPH | AWS |
|----------------------------------------------------------------------|-------------------------------------------------------------|------|-----|
| ls | list all buckets | O | O |
| ls s3://_bucket_ | list objects in the bucket | O | O |
| ls s3://_bucket_/_prefix_ | list objects match prefix in the bucket | O | O |
| ll | list all objects details (storage class, modify time, etag) | O | O |
| ll s3://_bucket_ | list objects detail in the bucket | O | O |
| ll s3://_bucket_/_prefix_ | list objects match prefix detail in the bucket | O | O |
| mb s3://_bucket_ | create bucket | O | O |
| rb s3://_bucket_ | delete bucket | O | O |
| put <file> s3://_bucket_/_object_ | upload the file | O | O |
| put <file> s3://_bucket_ | upload the file (file name as object name) | O | O |
| put test s3://_bucket_/_object_ | upload a test file sepcific object name | O | O |
| get s3://_bucket_/_object_ _file_ | download object | O | O |
| get s3://_bucket_/_object_ | download object in current folder | O | O |
| cat s3://_bucket_/_object_ | show the object content | O | O |
| del s3://_bucket_/_object_ [delete-marker:true] [secure-delete:true] | delete the object (with flag) | O | O |
| | delete-marker used in AWS | | O |
| | secure-delete used in BIGTERA(customized CEPH) | | |
| tag list s3://_bucket_/_object_ | list tag(s) to the object | O | O |
| tag ls s3://_bucket_/_object_ | list tag(s) to the object | O | O |
| tag add s3://_bucket_/_object_ _key1_=_value1_ [_key2_=_value2_] ... | add tag(s) to the object | O | O |
| tag put s3://_bucket_/_object_ _key1_=_value1_ [_key2_=_value2_] ... | add tag(s) to the object | O | O |
| tag del s3://_bucket_/_object_ | remove tag(s) from the object | O | O |
| tag rm s3://_bucket_/_object_ | remove tag(s) from the object | O | O |
| /uri?query | give the orignal url | O | O |

#### Table 2: List commands will send more than one http request
| HIGH LEVEL COMMAND | INTEGRATE FUNCTIONS | CEPH | AWS |
|--------------------|------------------------------------------------------------------------------------------|------|-----|
| la | list all objects | O | O |
| info s3://_bucket_ | acl(ceph, aws), location(ceph, aws), versioning(ceph, aws), uploads(ceph), version(ceph) | O | O |

#### Table 3: List commands only for CEPH with system keys
| COMMAND | FUNCTION |
|---------------------|-----------------------|
| usage s3://_bucket_ | show the bucket usage |


#### Table 4: List commands only for CEPH with system keys
| SHELL SETTING COMMAND | FUNCTION |
|-------------------------------|------------------------------------------|
| s3\_type [ceph/aws/aws4/aws2] | change the api for different S3 providor |
| s3\_type [ceph/aws/aws4/aws2] | change setting for different S3 providor |
| format [xml/json] | change the format client request |
| log [trace/debug/info/erro] | change the log level |
| | - Info : for Http header and body |
| | - debug: for auth signature hash info |
Expand All @@ -67,6 +72,7 @@ Download the link as follow and unzip
| Ctrl + d | logout and reselect user |


#### Table 5: The default format of S3 type
| S3 TYPE | AUTH TYPE | FORMAT | URL STYLE |
|---------|-----------|--------|----------------------|
| ceph | aws4 | json | path-style |
Expand Down
17 changes: 12 additions & 5 deletions src/main.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -444,19 +444,25 @@ USAGE:
{1}
list all buckets
{1} {2}
{1} s3://{2}
list all objects of the bucket
{1} s3://{2}/{40}
list objects with prefix in the bucket
{39}
list all object detail
{39} {2}
{39} s3://{2}
list all objects detail of the bucket
{3} {2}
{39} s3://{2}/{40}
list detail of the objects with prefix in the bucket
{3} s3://{2}
create bucket
{4} {2}
{4} s3://{2}
delete bucket
{5} {6} s3://{2}/{7}
Expand Down Expand Up @@ -566,7 +572,8 @@ USAGE:
"list".bold(),
"usage".bold(),
"info".bold(),
"ll".bold() //39
"ll".bold(),
"<prefix>".cyan(), //40
);
} else {
println!(
Expand Down

0 comments on commit e1bf918

Please sign in to comment.