IPFS API with verification in enclave

[brenzi]

implement an API to ipfs which enables the STF to:

• fetch a particular ipfs uri.
  • the content can be fetched by the untrusted worker, but the content multihash shall be verified inside the enclave
• (optional) write content to ipfs

Gitcoin Bounty

design approach

• please see: "a first version" comment below. not the most elegant, but pretty straight-forward.
• Alternatively, you may suggest better approaches (like i.e. querying the IPFS node directly from within the enclave - might be less effort and more elegant (caveat: rust-ipfs-api doesn't support no_std yet!).

Please describe your preferred approach in this thread prior to implementing it!

dev environment

You can use our docker to emulate SGX in SW mode, so you don't need SGX enabled HW for this task

acceptance tests:

• CI test passes
• unit tests covering all introduced functions pass. in particular:
  • enclave STF queries IPFS content and fails if content isn't correct, passes if correct (matching IPFS hash)
  • must also work for contents larger than 256kB
• integration test against real IPFS node passes (with similar tests to above)

[brenzi]

helpful resources:

How IPFs hash is created out of a DAG of 256kB chunks:
https://medium.com/swapynetwork/generating-ipfs-multihash-offline-2edb2618b93b

IPFS node in rust:
https://github.com/ipfs-rust/rust-ipfs

IPFS http API in rust:
https://github.com/ferristseng/rust-ipfs-api

[brenzi]

A first version could do the following:

1. run standalone IPFS node
2. enclace can tell worker to fetch a file from IPFS synchronously
3. worker fetches file via http request to IPFS node and stores it to disk (file name = IPFS hash)
4. when the host-call returns, the enclave reads the file from disk into memory
5. the enclave recreates the IPFS hash from the file contents and verifies that it matches the expected hash it requested

fetching IPFS content is already implemented. However, passing the content as a function return value might not scale to larger files. Moreover, the contents is not verified based on its ipfs hash

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 400.0 DAI (400.0 USD @ $1.0/DAI) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $87,762.30 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 266 years, 5 months from now.
Please review their action plans below:

1) developerfred has applied to start work (Funders only: approve worker | reject worker).

I already have previous experience with substrate and ipfs i would like to work on this issue.
2) whalelephant has been approved to start work.

Hi,

I think I would follow the first version approach (since not comfortable / not sure how much work is required for using no_std for the rust-ipfs-api).

High level work plan:

• environment set up (to get familiar with substraTEE and the docker SGX)
• understand /+ solve the limitations of ocall_read_ipfs on larger content (>256kB)
• worker write file to disk and enclave reads it into memory
• enclave create DAG from the chunks in memory and create the multiHash
• decode CID to multiHash to verify

Learn more on the Gitcoin Issue Details page.

[brenzi]

@x5engine: could you please share your brief work plan? How would you approach this?

[Web3Foundation]

@brenzi @x5engine Any updates regarding working on this issue?

[x5engine]

Hey guys, I will send implementation I did after tomorrow, we made a good implementation for this.

[brenzi]

@x5engine please see above: you haven't been assigned yet, we asked you for a sketch on how you will approach this. We'd like to review your design before we approve

[x5engine]

hey @brenzi sorry for the confusion, well I would plan to query the ipfs Node directly to get the has and save the file to the system then when requested I'd check the checksum of the file with the metadata checksum.

so, in other words, it should compute the checksum prior to uploading it

here something like it filecoin-project/venus#1925

[brenzi]

@Web3Foundation please assign @x5engine to work on this issue. The work plan is not very specific yet, but a more detailed design should be paid work I think.

@x5engine: You mentioned "uploading". Be aware that our task asks for a read interface, not a write interface. We want to fetch IPFS content from within the enclave STF. In order to avoid frustration, please provide us with a more detailed design first before implementing it. Where in our code would you do what?