weaviate-client fails to bundle on Cloudflare Workers due to http2 dependency

[pastelcolors]

Description

I am trying to use the weaviate-client library in a Cloudflare Worker, but I am encountering issues related to the http2 module. The error message suggests that the http2 package is built into Node.js and is not available in the Cloudflare Workers environment.

Context

I have added the following configuration to my wrangler.toml file to enable Node.js compatibility:

compatibility_date = "2023-12-01"
node_compat = true

However, even with this configuration, I am still facing the http2 related error.

Steps to Reproduce

1. Set up a new Cloudflare Worker project.
2. Install the weaviate-client library using npm or yarn.
3. Configure wrangler.toml with the appropriate compatibility_date and node_compat settings.
4. Write code that uses the weaviate-client library to make requests to a Weaviate instance.
5. Attempt to run the Worker using wrangler dev.

Expected Behavior

The Cloudflare Worker should be able to use the weaviate-client library to make requests to a Weaviate instance without any issues related to the http2 module.

Actual Behavior

The Worker fails to execute due to the missing http2 module, which is not available in the Cloudflare Workers environment.

✘ [ERROR] Could not resolve "http2"

    ../../node_modules/.pnpm/@[email protected]/node_modules/@grpc/grpc-js/build/src/transport.js:20:22:
      20 │ const http2 = require("http2");
         ╵                       ~~~~~~~

  The package "http2" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

Possible Solution

One potential solution is to use the native fetch API provided by Cloudflare Workers instead of relying on the http2 module used by the weaviate-client library.

Additional Information

• I'm using Weaviate Cloud
• weaviate-client library version: 3.0.0-rc.3

[tsmith023]

Hi @pastelcolors, the [email protected] library makes use of Weaviate's new gRPC API and so depends on the HTTP/2 protocol used by gRPC as a hard requirement. It is therefore only usable in a server-side environment, e.g. node.

Yours is a good use-case, however, and we will bear it in mind when we develop an alternative library that has the same API surface as [email protected] only depending on the GraphQL API under-the-hood instead. Such a library should then be usable in the browser and in serverless environments that do not support the HTTP/2 protocol.

As such, I will leave this issue open as a description of this limitation until the alternative library is released as a solution!

P.S. Are you able to reconfigure your Cloudflare Worker to be Node.js compatible? (I do not have experience with Cloudflare Workers so this suggestion may be unhelpful!)