Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Can not import .wasm (by @prisma/client/wasm, and other wasm-pack generated) #139

Open
mizchi opened this issue Nov 21, 2024 · 4 comments
Labels
bug Something isn't working triage

Comments

@mizchi
Copy link

mizchi commented Nov 21, 2024

Describe the bug

Opennext and @prisma/client/wasm do not work by import *.wasm.

Prisma does not work on opennext.

$ npm install prisma @prisma/client -D
$ npx prisma --init
# ... edit to use @prisma/client/wasm
$ npx next build
$ npx cloudflare # it works yet
$ wrangler deploy # fail at wrangler's  esbuild
   Creating an optimized production build ...
  Compiled successfully
  Linting and checking validity of types    
  Collecting page data    
Error occurred prerendering page "/". Read more: https://nextjs.org/docs/messages/prerender-error
PrismaClientKnownRequestError: 
Invalid `prisma.post.findMany()` invocation:


Unknown file extension ".wasm" for /home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/.prisma/client/query_engine_bg.wasm
    at dr.handleRequestError (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:21:7567)
    at dr.handleAndLogRequestError (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:21:6875)
    at dr.request (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:21:6565)
    at async u (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:30:9560)
    at async I (/home/mizchi/mizchi/next-memo/.next/server/app/page.js:1:39579)
Export encountered an error on /page: /, exiting the build.
  Static worker exited with code: 1 and signal: null
node:internal/errors:983
  const err = new Error(message);

Vite and webpack can handle it but wrangler's esbuild & workerd can not

Steps to reproduce

  • setup prisma
  • import @prisma/client/wasm
import { PrismaClient } from "@prisma/client/wasm";
import { PrismaD1 } from "@prisma/adapter-d1";
import { getCloudflareContext } from "@opennextjs/cloudflare";

export interface Env {
  DB: D1Database;
}

export async function getPrismaClient() {
  try {
    const ctx: any = await getCloudflareContext();
    return new PrismaClient({ adapter: new PrismaD1(ctx.env.DB) });
  } catch (error) {
    console.error("Error getting Prisma client", error);
    throw error;
  }
}

npx wrangler deploy

Expected behavior

@prisma/client works on opennext.

@opennextjs/cloudflare version

0.2.1

Node.js version

22.x

Wrangler version

3.88.0

next info output

Operating System:
  Platform: linux
  Arch: x64
  Version: #1 SMP Tue Nov 5 00:21:55 UTC 2024
  Available memory (MB): 64163
  Available CPU cores: 16
Binaries:
  Node: 22.9.0
  npm: 10.8.3
  Yarn: N/A
  pnpm: 9.9.0
Relevant Packages:
  next: 15.0.3 // Latest available version is detected (15.0.3).
  eslint-config-next: 15.0.3
  react: 18.3.1
  react-dom: 18.3.1
  typescript: 5.6.3
Next.js Config:
  output: N/A


### Additional context

This `.wasm`  import style is generated by Rust's `wasm-pack`
@mizchi mizchi added bug Something isn't working triage labels Nov 21, 2024
@mizchi mizchi changed the title [BUG] Can not import .wasm (by @prisma/client) [BUG] Can not import .wasm (by @prisma/client/wasm) Nov 21, 2024
@mizchi mizchi changed the title [BUG] Can not import .wasm (by @prisma/client/wasm) [BUG] Can not import .wasm (by @prisma/client/wasm, and other wasm-pack generated) Nov 21, 2024
@wmadden
Copy link

wmadden commented Nov 29, 2024

Hi @mizchi, it looks like your issue is caused by wrangler which by default will delete any .wasm files when you publish.

Others have solved this problem by adding configuration to wrangler.toml - read more about it here.

@vicb
Copy link
Contributor

vicb commented Nov 29, 2024

Good point, can you try adding find_additional_modules=true in your wrangler.toml?
It will add the the .wasm files as additional modules.

@mizchi
Copy link
Author

mizchi commented Dec 3, 2024

thx @vicb @wmadden

I used those links to verify this by rewriting node_moudules directly at hand.

First I needed to examine the prisma code generation and runtime steps.

  • Code generation process
    • Running npx prisma generate executes @prisma/client/generator-build/index.js by prisma's CLI, and @prisma/client/.prisma/client/* generates Assemble the client code.
    • Copy @prisma/client/.prisma/client/query_engine_wasm.bg according to Prisma's generator(postgres/mysql/sqlite)
  • Runtime
    • import {PrismaClient} from @prisma/client/wasm resolves @prisma/client/.prisma/client/wasm.js(cjs)
    • Initialize query_engine_bg.wasm as binary by fs.readFileSync with return new WebAssembly.Module(queryEngineWasmFileBytes).

https://github.com/prisma/prisma/blob/b6fe641510af621b30eb183076d98b61cdfe9c3d/packages/client/src/generation/utils/buildGetQueryEngineWasmModule.ts#L13-L21

I found some problems here.

  • According to this document workers need to initialize wasm in global scope before the fetch handler
  • prisma/client resolves "./wasm.js" as cjs, so imports in global scope cannot resolve it
  • The wrangler wasm_modules cannot be used in ESM mode. But the output of opennext always expects ESM
  • Webpack: opennext pre-processes next as webpack first, so .wasm cannot be resolved at this stage. you need to delay resolving .wasm until wrangler esbuild.

This is my trial and error. It hasn't completely worked yet, but it worked on hand to the point where it bypasses the initialization of wasm.

First, in next.config.js, prevent .wasm from loading.

    config.plugins.push(
      new webpack.IgnorePlugin({
        contextRegExp: /\\.wasm$/,
        resourceRegExp: /\\.wasm$/, }
      })
    );

At prisma/client/wasm runtime, modify the code generator to resolve from global variables.

diff --git a/generator-build/index.js b/generator-build/index.js
index f0adad1914bb059450d36d8df69acd57f9c95750..056bf25d709c0ca12f0b748d58f2156f54fadec9 100644
--- a/generator-build/index.js
+++ b/generator-build/index.js
@@ -7753,10 +7753,7 @@ function buildQueryEngineWasmModule(wasm, copyEngine, runtimeNameJs) {
     return `config.engineWasm = {
       getRuntime: () => require('. /query_engine_bg.js'),.
       getQueryEngineWasmModule: async () => {
- const queryEngineWasmFilePath = require('path').join(config.dirname, 'query_engine_bg.wasm')
- const queryEngineWasmFileBytes = require('fs').readFileSync(queryEngineWasmFilePath)
-fs'.      
- return new WebAssembly.Module(queryEngineWasmFileBytes)
+ return __PRISMA_BINARY;
       }
     }`;
   }
@@ -7764,9 +7761,7 @@ function buildQueryEngineWasmModule(wasm, copyEngine, runtimeNameJs) {
     return `config.engineWasm = {
   getRuntime: () => require('. /query_engine_bg.js'),.
   getQueryEngineWasmModule: async () => {
- const loader = (await import('#wasm-engine-loader')).default
- const engine = (await loader).default
- return engine 
+ return __PRISMA_BINARY;
   }
 }`;
   }

With this in place, copy . /query_engine_bg.wasm directly down into .workers-next and insert the following code into the .workers-next/index.mjs entry point for workers.

import wasm from ". /query_engine_bg.wasm";
globalThis.__PRISMA_BINARY = await WebAssembly.instantiate(wasm);
// ...original

This should now work, at least on the Workers runtime.

However, we could not verify that this actually works. The reason is that it fails in the prerender phase of the Next App Router. The prerender executes .next directly in node.js, so on the contrary, it can no longer resolve the __PRISMA_BINARY in this patch.

There may be a better way. I will experiment some more.

@vicb
Copy link
Contributor

vicb commented Dec 3, 2024

Thanks for the detailed analysis!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

3 participants