Emscripten support

[joemarshall]

Summary

I added a discussion for this ages back but there's been no input, so I've written it (because I was contracted to do the work anyway, so I might as well contribute it upstream). This PR adds support for running in emscripten / webassembly platforms, where all network connections go via the browser.

Currently in progress, but tests okay locally, so I've opened this to check the CI changes, I've got to update docs also.

Checklist

• [X ] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.

[tomchristie]

This is really interesting, thanks... ☺️

I've taken a bit of a look at the ecosystem here, tho am going to need a bit more orientation... Would it make sense to document an example of how to write an HTML page that includes a Python REPL with httpx imported and available?

[joemarshall]

@tomchristie I put in the PR that makes import ssl optional now (#3385 )

I updated this PR so it follows on from that PR.

How this PR works now is it moves _transports.default into _transports.httpcore, which defines [Async]HTTPCoreTransport, adds an extra module _transports.jsfetch file which defines [Async]JavascriptFetchTransport. Then in _transports/__init__.py it adds an alias of HTTPTransport which goes to whichever HTTP backend is in use (i.e. httpcore by default, JS fetch on emscripten)

[joemarshall]

@tomchristie I updated this to follow the changes in master - I think #3385 is redundant now, as the ssl changes are minimal at this point.

[samuelcolvin]

I'll add that I've been playing with httpx on https://pydantic.run over the last few days, both sync and async, and apart from the prints reported in pyodide/pyodide#5381, it seems to be working well otherwise!

[joemarshall]

Oh darn, can't believe I missed those debug prints. Fixed now.

[samuelcolvin]

Thanks so much @joemarshall for fixing those.

@hoodmane or @joemarshall, I'm not sure what the process is (or I'd try to help), but please can we update pyodide to use the head of his PR to avoid those debug print statements confusing users.

@tomchristie anything stopping this being merged?

[hoodmane]

I think Joe Marshall already sent a pr to Pyodide to update it so when we make another release it will bring it in.

[samuelcolvin]

I'm getting this error when using openai in pyodide, looks like the openai SDK is assuming something is bytes when it's actually a memoryview

  File "/lib/python3.12/site-packages/openai/_streaming.py", line 147, in __aiter__
    async for item in self._iterator:
  File "/lib/python3.12/site-packages/openai/_streaming.py", line 160, in __stream__
    async for sse in iterator:
  File "/lib/python3.12/site-packages/openai/_streaming.py", line 151, in _iter_events
    async for sse in self._decoder.aiter_bytes(self.response.aiter_bytes()):
  File "/lib/python3.12/site-packages/openai/_streaming.py", line 302, in aiter_bytes
    async for chunk in self._aiter_chunks(iterator):
  File "/lib/python3.12/site-packages/openai/_streaming.py", line 314, in _aiter_chunks
    for line in chunk.splitlines(keepends=True):
                ^^^^^^^^^^^^^^^^
AttributeError: 'memoryview' object has no attribute 'splitlines'

Any chance the error is related to this PR?

it works fine locally, and it works fine when not using their streaming responses. Happy to give more details or create a separate issue if that helps?

[tomchristie]

anything stopping this being merged?

I would suggest not making any PR/changes to httpx, and start by demo'ing an Emscripten transport.