Mounting Custom Async File System

[MeydanOzeri]

Hi I'm new to emscripten, and i have to say the learning curve is steep, and at the moment the docs are not dated so it's hard to understand how to get something done that is not the usual or simple case, hopefully i can get help here.

My use case is to try to make git run on mobile apps by compiling libgit2 to web assembly using emscripten, there is a project of that on github: https://github.com/petersalomonsen/wasm-git

My issue is that in order for me to do it successfully i must use an async file system, the executing runtime on mobile (capacitor) only supports async file system functions, due to that i don't understand how to make it possible.

What i tried so far was to mount my custom created FS:

FS.mount(customFS, {}, '/path')

this mounts successfully using the default MEMFS, after that as expected it tries to invoke functions on my customFS.
But, if i try to include async operations it breaks as nothing awaits those calls, so i figured its immposible.

I saw talks in github on memfs and opfs, and thought that this might be the solution, but when i build using:
-s WASMFS -lopfs.js -s FORCE_FILESYSTEM i see there is no more a mount function on the FS object, im quite confused and cannot find any info on this, help would be appreciated.

Edit: now after updates there is mount function, but its different than how it was before wasmfs and I cannot get it to work, also i dont know if async is possible, any docs or info somewhere ?

[MeydanOzeri]

anyone ?

[sbc100]

@tlively who worked on the OPFS backend might have some pointers. IIRC it involved proxying the work to a worker that can block. My advice would be to look at how that is implemented.

BTW, where are you expecting the git data to actually live? (i.e. the data that normally lives in the .git directory?)

[MeydanOzeri]

Well it doesn't run in a browser in my use case, maybe i failed to mention that.

It runs on electron in PC, and a native mobile app built with capacitor, these frameworks allow the file system access.
The issue is that git is not supported in mobile, and therefore i'm trying to compile libgit which is written in C to wasm so it will be able to run git on mobile.
If capacitor would allow sync file access i would be happy, unfortunately its not the case.

[MeydanOzeri]

I didn't manage to find in the OPFS how to achieve what i want, any directions or specific places i should look into ?

[tlively]

The OPFS backend uses async file system APIs internally, but it presents a synchronous interface to its users. @MeydanOzeri, is that similar to your situation, or do have the opposite problem where you want to provide an asynchronous interface to the file system users? An asynchronous file system interface is something we've considered doing for WasmFS and the OPFS backend, but we haven't gotten to it yet.

Have you looked into using Asyncify to get an asynchronous interface to your Wasm module? https://emscripten.org/docs/porting/asyncify.html

[MeydanOzeri]

I think the second one, i need the compiled emscripten js file to make a call to my async file system api.
I am not sure if asyncify solves my problems as the fs is created synchronously, it doesn't await any of my functions if they are async, unless it can make MEMFS calls async somehow.

[tlively]

Ok, if you need emscripten to call lower-level async file system APIs provided by your runtime, then that's the exact situation we have with the WasmFS OPFS backend, so you should be able to do something similar. See opfs_backend.cpp for how it using the proxying API to synchronously wait for asynchronous JS work (from library_wasmfs_opfs.js) to be run in a different thread.