TFHE-rs for Web Assembly

[patriciaOrtuno28]

Hi!
I'm opening this issue to ask about the progress of the migration of Concrete's libraries to Web Assembly. I intend to use it with wasm-bindgen, but, at the moment, Concrete only supports x86_64 arch from what I've seen.
Thank you in advance.

[agnesLeroy]

Hi @patriciaOrtuno28! As you noticed Concrete does not offer Web Assembly support as of now, and I cannot really say when it will be set up: this will probably take some time. However, Concrete relies on Concrete-core, which itself is now able to target various platforms. It comes with a Web Assembly API for operations performed on the Client side (encryption, decryption, key creation basically) in the concrete-core-wasm crate. Maybe this could be a solution for you in the meanwhile?
Cheers,
Agnès

[patriciaOrtuno28]

Hi @patriciaOrtuno28! As you noticed Concrete does not offer Web Assembly support as of now, and I cannot really say when it will be set up: this will probably take some time. However, Concrete relies on Concrete-core, which itself is now able to target various platforms. It comes with a Web Assembly API for operations performed on the Client side (encryption, decryption, key creation basically) in the concrete-core-wasm crate. Maybe this could be a solution for you in the meanwhile?
Cheers,
Agnès

Hi! Okey, that's really helpful. As Concrete relies on Concrete core, I understand I could use concrete-core-wasm on the frontend for key generation, encryption and decryption and I could have a Rust API working with Concrete boolean (v0.1.0) to manage operations?
Thank you!

[agnesLeroy]

Yes I hope this would be possible, although maybe not very easy to set up. If you try it out we'll be interested to know what are the issues you have, and we'll be happy to help. 🙂

[patriciaOrtuno28]

Yes I hope this would be possible, although maybe not very easy to set up. If you try it out we'll be interested to know what are the issues you have, and we'll be happy to help. 🙂

I already have the API working with Concrete Boolean, connected to a MongoDB database.
We can leave this issue open for when I try to add the concrete-core-wasm dependencies to the frontend. It can be interesting for other users who are trying to make Concrete work for 3 layer architectures as well.
Best regards.

[patriciaOrtuno28]

Hi @agnesLeroy, sorry for the delay. I already have the backend working with Concrete Boolean as I mentioned, and I've been working on the Frontend to achieve the 3 layer arquitecture.
With concrete-core-wasm I can build a pkg folder with the required TypeScrypt and Javascript files for a node-module, to then install into a React project. The maximum version of node I could make this work with was [email protected], and then run the React app with [email protected] (Avoiding v5 due to Polyfill unsupported errors).
The only mistake I encountered, probably due to versions differing was the import { TextDecoder, TextEencoder } from 'util';, which I had to substitute for the following:

const TextEncodingPolyfill = require('text-encoding');
Object.assign(global, {
  TextEncoder: TextEncodingPolyfill.TextEncoder,
  TextDecoder: TextEncodingPolyfill.TextDecoder,
});

The node package can be built and compiled without an issue. The problem is that it has been designed for a node API. After the last definition of the exported functions, there is this line: const bytes = require('fs').readFileSync(path);. Te 'fs' package doesn't work in browsers, which led me to think the concre-core-wasm crate has been only designed for backend purposes. To try to adjust it to the browser's rules I included "browser": { "fs": false, "path": false} in the package.json file. Even with that change the error appearing in the browser's console is still "Uncaught TypeError: webpack_require(...).readFileSync is not a function", which means it keeps trying to read the 'fs' library. I have also tried other npm packages that emulate 'fs''s behavior, but none of them seem to do the trick.

Having explained the problem, I was wondering if anyone has tried to apply this crate to a Frontend, and how they got around the 'fs' issue for the browser. Thank you in advance.

[wyatt-howe]

I can say it is feasible to at least get the encryption/decryption methods compiled and working. (I worked it out last summer - sadly in a private repo.) The bootstrapping on the other hand is going to require a lot of reimplementing x86_64 dependencies and optimizations, but doing that work in the browser is going to be impractical anyway.

[patriciaOrtuno28]

I can say it is feasible to at least get the encryption/decryption methods compiled and working. (I worked it out last summer - sadly in a private repo.) The bootstrapping on the other hand is going to require a lot of reimplementing x86_64 dependencies and optimizations, but doing that work in the browser is going to be impractical anyway.

Hi! I just need the encryption and decryption, nothing else. Could you possibly share how you got around the 'fs' library not being available for the browser issue? Or any tip on how to achieve it. Thank you in advance!

[wyatt-howe]

I can say it is feasible to at least get the encryption/decryption methods compiled and working. (I worked it out last summer - sadly in a private repo.) The bootstrapping on the other hand is going to require a lot of reimplementing x86_64 dependencies and optimizations, but doing that work in the browser is going to be impractical anyway.

Hi! I just need the encryption and decryption, nothing else. Could you possibly share how you got around the 'fs' library not being available for the browser issue? Or any tip on how to achieve it. Thank you in advance!

It seems I actually did publish the WebAssembly part of a Concrete project to my personal account: https://github.com/wyatt-howe/concrete-web. I'm no longer familiar with the latest version of Concrete, and the linked project predates the official WebAssembly API (so I never had that 'fs' library issue). Again, it's old, and works for arithmetic circuits only (I didn't realize there's now a "Concrete Boolean"!), but perhaps it is useful still.

[patriciaOrtuno28]

I can say it is feasible to at least get the encryption/decryption methods compiled and working. (I worked it out last summer - sadly in a private repo.) The bootstrapping on the other hand is going to require a lot of reimplementing x86_64 dependencies and optimizations, but doing that work in the browser is going to be impractical anyway.

Hi! I just need the encryption and decryption, nothing else. Could you possibly share how you got around the 'fs' library not being available for the browser issue? Or any tip on how to achieve it. Thank you in advance!

It seems I actually did publish the WebAssembly part of a Concrete project to my personal account: https://github.com/wyatt-howe/concrete-web. I'm no longer familiar with the latest version of Concrete, and the linked project predates the official WebAssembly API (so I never had that 'fs' library issue). Again, it's old, and works for arithmetic circuits only (I didn't realize there's now a "Concrete Boolean"!), but perhaps it is useful still.

Hi! Sorry for the delay. I can't access the repository, I think it's private.

[leventdem]

Hi @agnesLeroy, sorry for the delay. I already have the backend working with Concrete Boolean as I mentioned, and I've been working on the Frontend to achieve the 3 layer arquitecture. With concrete-core-wasm I can build a pkg folder with the required TypeScrypt and Javascript files for a node-module, to then install into a React project. The maximum version of node I could make this work with was [email protected], and then run the React app with [email protected] (Avoiding v5 due to Polyfill unsupported errors). The only mistake I encountered, probably due to versions differing was the import { TextDecoder, TextEencoder } from 'util';, which I had to substitute for the following:

const TextEncodingPolyfill = require('text-encoding');
Object.assign(global, {
  TextEncoder: TextEncodingPolyfill.TextEncoder,
  TextDecoder: TextEncodingPolyfill.TextDecoder,
});

The node package can be built and compiled without an issue. The problem is that it has been designed for a node API. After the last definition of the exported functions, there is this line: const bytes = require('fs').readFileSync(path);. Te 'fs' package doesn't work in browsers, which led me to think the concre-core-wasm crate has been only designed for backend purposes. To try to adjust it to the browser's rules I included "browser": { "fs": false, "path": false} in the package.json file. Even with that change the error appearing in the browser's console is still "Uncaught TypeError: webpack_require(...).readFileSync is not a function", which means it keeps trying to read the 'fs' library. I have also tried other npm packages that emulate 'fs''s behavior, but none of them seem to do the trick.

Having explained the problem, I was wondering if anyone has tried to apply this crate to a Frontend, and how they got around the 'fs' issue for the browser. Thank you in advance.

Hello, did you try to build the wasm package for the "--target web" version and not "--target node-js" using wasm-pack (explained here )?

Using the web is more adapted for the frontend.

[patriciaOrtuno28]

Hi @agnesLeroy, sorry for the delay. I already have the backend working with Concrete Boolean as I mentioned, and I've been working on the Frontend to achieve the 3 layer arquitecture. With concrete-core-wasm I can build a pkg folder with the required TypeScrypt and Javascript files for a node-module, to then install into a React project. The maximum version of node I could make this work with was [email protected], and then run the React app with [email protected] (Avoiding v5 due to Polyfill unsupported errors). The only mistake I encountered, probably due to versions differing was the import { TextDecoder, TextEencoder } from 'util';, which I had to substitute for the following:

const TextEncodingPolyfill = require('text-encoding');
Object.assign(global, {
  TextEncoder: TextEncodingPolyfill.TextEncoder,
  TextDecoder: TextEncodingPolyfill.TextDecoder,
});

The node package can be built and compiled without an issue. The problem is that it has been designed for a node API. After the last definition of the exported functions, there is this line: const bytes = require('fs').readFileSync(path);. Te 'fs' package doesn't work in browsers, which led me to think the concre-core-wasm crate has been only designed for backend purposes. To try to adjust it to the browser's rules I included "browser": { "fs": false, "path": false} in the package.json file. Even with that change the error appearing in the browser's console is still "Uncaught TypeError: webpack_require(...).readFileSync is not a function", which means it keeps trying to read the 'fs' library. I have also tried other npm packages that emulate 'fs''s behavior, but none of them seem to do the trick.
Having explained the problem, I was wondering if anyone has tried to apply this crate to a Frontend, and how they got around the 'fs' issue for the browser. Thank you in advance.

Hello, did you try to build the wasm package for the "--target web" version and not "--target node-js" using wasm-pack (explained here )?

Using the web is more adapted for the frontend.

I didn't know that was an option. Thank you! It works fine with a React App.

[lastmjs]

Hi, I'm building on top of the Internet Computer blockchain which is a Wasm environment, and to get these libraries working we'll need a clean wasm32-unknown-unknown compilation.

As you work towards Wasm compatibility, please don't assume a browser environment. Like I said, a clean wasm32-unknown-unknown with no dependencies on the browser would be required for us in the Internet Computer ecosystem.

[IceTDrinker]

Hi, I'm building on top of the Internet Computer blockchain which is a Wasm environment, and to get these libraries working we'll need a clean wasm32-unknown-unknown compilation.

As you work towards Wasm compatibility, please don't assume a browser environment. Like I said, a clean wasm32-unknown-unknown with no dependencies on the browser would be required for us in the Internet Computer ecosystem.

Reproducing the reply from the Zama discourse (https://community.zama.ai/t/wasm32-unknown-unknown/342/2) here.

Edit: the binding to come is focused on the client side only (encryption, decryption, keygen and serialization).

Hello,

As far as I can tell (as I've been working on the wasm binding lately) is that wasm-pack builds a binary targetting wasm32-unknown-unknown using rustc and then generates some javascript on top.

However some code that compiles to wasm will actually trigger some asserts/aborts. It is for example the case for the code benchmarking the fft algorithms to choose the best one, as std::time is not available for wasm32-unknown-unknown. I discovered that the hard way. Same thing for multithreaded code via rayon, some init is not done as wasm32 lacks some threading specification "to remain widely compatible" according to some sources.

Finally it's a 32 bits target, as the wasm32 indicates. Some keys won't fit in memory (as I've had to learn again the hard way). In practice it seems implementations use i32 for indexing memory meaning you "only" have 2GB of RAM to work with, which can prove too small for FHE and big parameter sets.

If you have other questions feel free to ask them here 🙂

Cheers

[IceTDrinker]

this is issue is ancient and the question has resurfaced here #1034

Closing this issue in favor of the more recent one