feat(site-builder): add pre-stored blobs support

[Tzal3x]

This is just a transfer of the work done by @giac-mysten in this branch.

Summary

It is now possible inside the ws-resources.json to specify pre-stored blobs (resources) in order for them to not be re-uploaded on walrus.

For example, to specify an already-uploaded index.html to include inside your walrus site, you have to define the following:

{
"pre_built":[
      {
         "path":"/index.html",
         "headers":{
            "Cache-Control":"no-cache",
            "Content-Encoding":"gzip",
            "Content-Type":"application/json"
         },
         "blob_id":"LVLk9VSnBrEgQ2HJHAgU3p8IarKypQpfn38aSeUZzzE",
         "blob_hash":"844821ccafb8bc8e9f6dd69afd6bce1bdd0993b141acdb9a278765fe1ed360a2"
      }
   ]
}

Note: to calculate the blob hash of a file one needs to run sha256sum <dir-of-file>

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
walrus-sites-sp	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 19, 2024 3:13pm
walrus-sites-sp-testnet	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 19, 2024 3:13pm
walrus-sites-sw	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 19, 2024 3:13pm
walrus-sites-sw-testnet	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 19, 2024 3:13pm

[codspeed-hq]

CodSpeed Performance Report

Merging #306 will not alter performance

_{Comparing ta/pre-stored-blobs--continuation-from-giac-branch (7f01010) with main (416af2b)}

Summary

✅ 3 untouched benchmarks

[Tzal3x]

Currently facing the following error:

Error: running the command exited with error: 2024-11-19T16:38:32.228358Z  INFO walrus_service::client::cli::args: running in JSON mode
2024-11-19T16:38:32.228396Z  INFO walrus: client version: 1.0.0-1a9dc8c2a8e8
2024-11-19T16:38:32.228435Z  INFO walrus_service::client::cli: using Walrus configuration from '/Users/alex/.walrus/client_config.yaml'
2024-11-19T16:38:32.228527Z  INFO walrus_service::common::utils: using Sui wallet configuration from '/Users/alex/.sui/sui_config/client.yaml'
2024-11-19T16:38:32.720053Z  WARN sui_sdk::wallet_context: Client/Server api version mismatch, client api version : 1.35.0, server api version : 1.38.0
[warn] Client/Server api version mismatch, client api version : 1.35.0, server api version : 1.38.0
2024-11-19T16:38:33.281271Z  INFO walrus_service::client::cli::runner: Storing file  as blob on Walrus
Error: unable to read blob from '': No such file or directory (os error 2)

This should have been already fixed by Giacomo in a previous PR but it's possible that I reintroduced it while transferring the changes from the old branch to this one.

[giac-mysten]

Hey @Tzal3x . There are still several hacks in this code that were added to get the feature out quickly. I would suggest we rework this before merging it. I can do it if I find the time, but feel free to grab it.