Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Netty based s3 API in worker #17661

Merged
merged 52 commits into from
Aug 3, 2023
Merged

Implement Netty based s3 API in worker #17661

merged 52 commits into from
Aug 3, 2023

Commits on Jun 20, 2023

  1. support headObject in netty server

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    ef8afa8 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2023

  1. getObject API demo (can work)

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    83d77be View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    2a32752 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2023

  1. refactoring some method and fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 25, 2023
    Configuration menu
    Copy the full SHA
    2a656d8 View commit details
    Browse the repository at this point in the history

  2. support listAllBucket

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 25, 2023
    Configuration menu
    Copy the full SHA
    ef0bc35 View commit details
    Browse the repository at this point in the history

  3. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 25, 2023
    Configuration menu
    Copy the full SHA
    12fba3b View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2023

  1. support listObject

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    36889a5 View commit details
    Browse the repository at this point in the history

  2. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    eba6140 View commit details
    Browse the repository at this point in the history

  3. support redirect in netty s3(WIP)

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    17d33ae View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2023

  1. fix some bugs of the Chunked read in GetObject

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 27, 2023
    Configuration menu
    Copy the full SHA
    452ff4d View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2023

  1. fix some bugs of the Chunked read in GetObject

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    a745fbc View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2023

  1. support putObject and headBucket

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    a8eead8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    c4f08ed View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2023

  1. fix some bugs of getObjects

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    addfafe View commit details
    Browse the repository at this point in the history

  2. add access log for netty s3 api

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    8e4994d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    4af6f2b View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. get content hash from FileInfo

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    9138fc2 View commit details
    Browse the repository at this point in the history

  2. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    8b373a8 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    1d558b0 View commit details
    Browse the repository at this point in the history

  4. enable AuditLogWriter

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    22ef2a3 View commit details
    Browse the repository at this point in the history

  5. fix unit test

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    b198d16 View commit details
    Browse the repository at this point in the history

  6. fix unit test

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    63f25af View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2023

  1. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    95ee41f View commit details
    Browse the repository at this point in the history

  2. support chunked encoding in create object

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    bcc769b View commit details
    Browse the repository at this point in the history

  3. change the way to get etag of file

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    83d383d View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2023

  1. support create/delete bucket

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    d9c76ee View commit details
    Browse the repository at this point in the history

  2. support copyObject(WIP)

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    ee6ec51 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. support copyObject

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    300918b View commit details
    Browse the repository at this point in the history

  2. support deleteObject

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    2c605a0 View commit details
    Browse the repository at this point in the history

  3. fix some checkstyle issues

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    10f6aeb View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    35c15f4 View commit details
    Browse the repository at this point in the history

  5. fix some checkstyle issues

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    f55b2a7 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2023

  1. support heavyPool and lightPool

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    4e8a268 View commit details
    Browse the repository at this point in the history

  2. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    3d58e7f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    ef45c9a View commit details
    Browse the repository at this point in the history

  4. pull the latest main

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    36b691d View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. fix review comments

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    6f58767 View commit details
    Browse the repository at this point in the history

  2. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    b96e107 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api

� Conflicts:
�	dora/tests/src/test/java/alluxio/client/rest/CreateBucketTest.java
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    909dc33 View commit details
    Browse the repository at this point in the history

  4. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    f036961 View commit details
    Browse the repository at this point in the history

  5. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    7eee172 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. fix async task issue

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    5dee107 View commit details
    Browse the repository at this point in the history

  2. fix async task issue

    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    7e87373 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. Don't support putObject && Add the switch to disable netty s3 server … 

    …by default
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    46b0420 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api

� Conflicts:
�	dora/minicluster/src/main/java/alluxio/multi/process/MultiProcessCluster.java
�	dora/tests/src/test/java/alluxio/client/rest/RestApiTest.java
    @Jackson-Wang-7
    Jackson-Wang-7 committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    9eda306 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. fix merge conflict

    @Jackson-Wang-7
    Jackson-Wang-7 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    6dab9b3 View commit details
    Browse the repository at this point in the history

  2. fix some review comments

    @Jackson-Wang-7
    Jackson-Wang-7 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    7787793 View commit details
    Browse the repository at this point in the history

  3. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    a204f4b View commit details
    Browse the repository at this point in the history

  4. fix checkstyle

    @Jackson-Wang-7
    Jackson-Wang-7 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    acd07f6 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    3bb1baf View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. Merge branch 'main' of github.com:Alluxio/alluxio into netty_based_s3… 

    …_api
    @Jackson-Wang-7
    Jackson-Wang-7 committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    a0ccd41 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. fix some review comments

    @Jackson-Wang-7
    Jackson-Wang-7 committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    c019b3d View commit details
    Browse the repository at this point in the history