Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WJ-1196] Add support for S3 presign URLs to upload blobs #1918

Draft
wants to merge 38 commits into
base: develop
Choose a base branch
from
Draft

[WJ-1196] Add support for S3 presign URLs to upload blobs #1918

wants to merge 38 commits into from

Commits on Sep 20, 2024

  1. Add new file section to deepwell config.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    3bcd2a0 View commit details
    Browse the repository at this point in the history

  2. Add file section to configuration.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    4c48597 View commit details
    Browse the repository at this point in the history

  3. Change field to seconds, not Duration.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    c3b4585 View commit details
    Browse the repository at this point in the history

  4. Begin BlobService::upload_url().

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    2013d08 View commit details
    Browse the repository at this point in the history

  5. Add blob_upload table.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    43c5c75 View commit details
    Browse the repository at this point in the history

  6. Add timestamp for partial upload pruning.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    1e59199 View commit details
    Browse the repository at this point in the history

  7. Start BlobService changes for presign URL system.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    9642ba4 View commit details
    Browse the repository at this point in the history

  8. Add file_pending migration and pending model.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    9484a86 View commit details
    Browse the repository at this point in the history

  9. Start upload code.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    44bce20 View commit details
    Browse the repository at this point in the history

  10. Rename file_pending -> blob_pending.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    892d751 View commit details
    Browse the repository at this point in the history

  11. Fix compilation.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    6c2bae4 View commit details
    Browse the repository at this point in the history

  12. [WIP] Start division between new and edit file uploads.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    d3c3f98 View commit details
    Browse the repository at this point in the history

  13. Add created_at column to blob_pending.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    46dfe30 View commit details
    Browse the repository at this point in the history

  14. Use find_by_id() instead of find().

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    7219b39 View commit details
    Browse the repository at this point in the history

  15. Run rustfmt.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    638495b View commit details
    Browse the repository at this point in the history

  16. Add FileRevisionService::create_pending().

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    2355013 View commit details
    Browse the repository at this point in the history

  17. Add FileRevisionService::get_first().

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    904d61f View commit details
    Browse the repository at this point in the history

  18. Update comment.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    066a8fe View commit details
    Browse the repository at this point in the history

  19. Rename structs.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    19d44cb View commit details
    Browse the repository at this point in the history

  20. Run rustfmt.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    2f3c28c View commit details
    Browse the repository at this point in the history

  21. Add proper StartFileUploadOutput struct.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    cfac455 View commit details
    Browse the repository at this point in the history

  22. Reword column clear again.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    1d683e3 View commit details
    Browse the repository at this point in the history

  23. Update comments.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    34379a3 View commit details
    Browse the repository at this point in the history

  24. Remove dead_code suppression.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    20308c4 View commit details
    Browse the repository at this point in the history

  25. Add TODOs for incomplete file pruning jobs.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    c4a3dac View commit details
    Browse the repository at this point in the history

  26. Add FileRevisionService::finish_upload(). 

    And complete FileService::finish_new_upload().
    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    6cf5fd3 View commit details
    Browse the repository at this point in the history

  27. Improve output of finish_new_upload().

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    a22bd48 View commit details
    Browse the repository at this point in the history

  28. Delete dummy structs.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    65a0b6c View commit details
    Browse the repository at this point in the history

  29. Rename types.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    0a9cbc7 View commit details
    Browse the repository at this point in the history

  30. Merge pending jobs.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    7f1c684 View commit details
    Browse the repository at this point in the history

  31. Stub out edits for now. 

    First need to figure out initial uploads, then will unify them in some
nice, generic way.
    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    064276a View commit details
    Browse the repository at this point in the history

  32. Fix build errors.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    2a99341 View commit details
    Browse the repository at this point in the history

  33. Fix CHECK constraints.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    df1c682 View commit details
    Browse the repository at this point in the history

  34. Implement file_upload_* API methods.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    d6876e7 View commit details
    Browse the repository at this point in the history

  35. Rename file creation structs.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    fd3c497 View commit details
    Browse the repository at this point in the history

  36. Rename upload API methods.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    70fbfd1 View commit details
    Browse the repository at this point in the history

  37. Remove unused struct.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    e1e3b75 View commit details
    Browse the repository at this point in the history

  38. Address warnings.

    @emmiegit
    emmiegit committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    08161de View commit details
    Browse the repository at this point in the history