Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#5779] feat(iceberg): add OSS support for IcebergRESTService docker image #6096

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

TungYuChiang
Copy link
Contributor

What changes were proposed in this pull request?

add OSS support for IcebergRESTService docker image

Why are the changes needed?

Fix: #5779

no

How was this patch tested?

run SQL with access Aliyun OSS data

@FANNG1
Copy link
Contributor

FANNG1 commented Jan 4, 2025

Could you add GRAVITINO_OSS_ROLE_ARN and test with oss-token providers? You should create an Aliyun role and grant it the privilege to access the OSS data.

Copy link
Collaborator

@orenccl orenccl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we also need to update docker-image-details.md

- apache/gravitino-iceberg-rest:0.8.0-incubating
  - Supports OSS and ADLS storage.

螢幕擷取畫面 2025-01-04 150814

@TungYuChiang
Copy link
Contributor Author

@FANNG1 Thank you for the suggestion. I have added the GRAVITINO_OSS_ROLE_ARN, and I also tested the oss-token provider on Alibaba Cloud. Below is the process and results:

  1. Configured gravitino-iceberg-rest-server.conf.template.
  2. Built and ran a custom Docker image.
  3. Started Spark SQL.
  4. Result:
Screenshot 2025-01-05 at 7 39 04 PM Screenshot 2025-01-05 at 7 39 37 PM

@FANNG1
Copy link
Contributor

FANNG1 commented Jan 5, 2025

LGTM, Could you also update the docker image document?

- apache/gravitino-iceberg-rest:0.8.0-incubating
  - Supports ADLS and OSS storage

update the iceberg rest service document

docker run -d -p 9001:9001 apache/gravitino-iceberg-rest:0.8.0-incubating

@TungYuChiang
Copy link
Contributor Author

@FANNG1 @orenccl Thank you for review, I have already fixed all comments.

Copy link
Collaborator

@orenccl orenccl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except for one comment. 😄

@@ -60,9 +60,10 @@ docker run --rm -d -p 9001:9001 apache/gravitino-iceberg-rest:0.7.0-incubating

Changelog

- apache/gravitino-iceberg-rest:0.7.0-incubating
- apache/gravitino-iceberg-rest:0.8.0-incubating
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add new log, instead of modify 0.7.0 to 0.8.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement] add OSS support for IcebergRESTService docker image
3 participants