Install channels without daphne

[ostcar]

Currently channels has daphne as dependency. So it is not possible to install channels on one machine without installing daphne with its dependencies.

The only place in code of channels, which needs daphne, is the runserver command, which kind of a development command that should not be used in production. So there is no need to require daphne as a requirment for a production setup.

Could it be possible to create a package, that you can install without installing daphne as well?

Our background is as follows: We have a django-channels app, that has be used also on windows from low-tech-users. But it is quite hard to install a current version of twisted on windows. So we build geiss as an replacement for daphne, which does not need twisted. But with the current package of channels, it is not possible to get rid of twisted, even when we don't want to use daphne.

But I think, that there are other use cases where someone wants to install channels without daphne. For example if you want to separate the worker on different containers/hardware then the protocol server.

[andrewgodwin]

This is a good point, but I also want to ensure that people who pip install channels still end up with a working runserver command, because if we change it to pip install channels[daphne] it's going to result in a lot more people being very confused.

Do you know of a way we could have a default dependency in Python that's possible to exclude?

[ostcar]

I have no idea if it is possible to have optional default dependencies in python.

Another way to solve this would be, if renamed the current package to something like channels-solo and removed daphne as dependency and created a new (meta-)package, which as daphne and channels-solo as dependency.

[proofit404]

My guess is that making runserver command implementation independent of Daphne code base will be the right decision. For example we can make Daphne itself depends on runserver at this moment and eventually move it totally into fully async twisted stack.

[andrewgodwin]

That will still destroy the first time user experience though. I even think having runserver soft-depend on daphne and print Please install daphne the first time is a bit annoying. Let me see if I can think of another way.

[proofit404]

Maybe I wasn't clean. runserver will work out of the box without installing Daphne. So first impression still be the same. But when you deploy to production you'll install Daphne explicitly.

[ostcar]

@andrewgodwin Did you find time to think about this issue?

My preference would be, if you created two packages. A package called "channels" that has no dependency to daphne and does not ship the runserver command. Another package called for example "channels-environment", ships the runserver command and has all the dependencies.

[andrewgodwin]

I have given it some thought, but there's not an ideal solution yet. I don't really want to make two packages as channels is already 5 packages as it is , but if I did, it would be something like channels-light and channels, so that pip install channels still does the expected thing.

It's possible this can be done with a single setup.py even, just with environment variables that switch to the -light build during the release process, but that still makes things a bit over-complicated.

The other part of the issue is that so far, you're the only person requesting the change, so it's quite low priority at the moment compared to other issues that are around.

[ThaJay]

I like the idea of runserver not depending on daphne because ws4redis does not run uwsgi directly, instead it runs runserver which does not work any more once channels is installed and runserver gets hijacked to use daphne instead. This makes it very difficult to run both channels and ws4redis (as long as our users need to get their apps up to date).

edit: we will just uninstall ws4redis

[rixx]

I like the idea of a separate package version (pip install channels[standalone]). This would be way more doable if setuptools would allow for one of the defined extras to be default, but there does not seem to be a way to achieve this. But I don't think that "Please install channels[standalone]" or a similar message is too much of a stretch, and I'd prefer it a lot over having to install a separate package.

[proofit404]

The only way I know how to do it is some think like have empty PyPI package called channels which depends on daphne and channels-standalone. Python package channels will be provided by channels-standalone.

[andrewgodwin]

The pip install channels[standalone] idea is that standalone would have runserver support? As in, when you run runserver, that's what prints if you installed non-standalone?

[rixx]

That was my idea, yes. I know that having a channels[bare] with channels as-is including runserver would be much nicer, but apparently that's just not possible with plain setuptools.

[creeon2003]

Great idea ...

[bastbnl]

@ostcar did you consider that you'll need to run manage.py runworkers in production in order to get the websockets running? Or did you find a way around that as well?

[ostcar]

Yes, I have to call manage.py runworkers but the runworker could be started without daphne. The the import statements in the files for runworker and runserver.

So I am not sure, if I understand your question correctly.

[bastbnl]

Thanks @ostcar. Your statement concerning manage.py in production got me wondering if you've found a way to run the workers in some other way.

My question was totally unrelated to daphne, indeed.

[andrewgodwin]

I haven't found a good solution to this even in the Channels 2.0 refactor, so I am going to close this as an effective WONTFIX. If someone can think of a clever solution that preserves pip install channels as working and installing Daphne, we can re-open.

[proofit404]

Maybe we can have two separate setup.py for classic channels and channels-bare packages. The second one is channels without dependencies and the first one depends on the second one and Daphne and ASGI and etc..

But I don't know if we really want to support this complexity.

[andrewgodwin]

I just don't think it's worth the overhead to maintain and publish two separate packages. If someone cannot install Daphne for some reason, they can download and build their own package from the source plus a patch easily.

[reorx]

Currently the solution workaround for me is to install channels with --no-deps option, and then install its dependencies without daphne separately:

pip install --no-deps channels
pip install asgiref

In practice I do use channels without daphne, because uvicorn seems to be a better choice than daphne, and since ASGI technology is evolving actively, I think make channels a pure library without server could becoming more and more considerable.

One thing also worth noticing is that daphne requires twisted, which is very bloated comparing to uvloop required by uvicorn...

[rneilson]

If I could make an argument to reconsider this issue: the problem with requiring daphne is its dependency on Twisted. At least the current version (20.3.0) requires a compiler, which when using eg uvicorn instead, and creating a container using eg python:3.8-slim, means extra build steps and bloat. The workaround above - installing channels without dependencies - is very much a workaround, not a solution.

[carltongibson]

So looking on the pip issue tracker, there's this closed issue, "Way to specify --no-deps option in requirements.txt?"

There's a suggested approach:

If you still wanted to maintain a single file, I don't think it would be hard to just pre-process your requirements.txt and invoke pip separately as suggested by @pfmoore above, like

grep -v -e --no-deps requirements.txt > requirements-with-deps.txt
grep -e --no-deps requirements.txt > requirements-without-deps.txt
pip install -r requirements-without-deps.txt --no-deps
pip install -r requirements-with-deps.txt

Looking at uvicorn, they've now got $ pip install uvicorn[standard] as the default, because you can't specify just the base package as an extra... — which is a bit 😬 — it would be nicer if there were a better approach.

Maybe we can think about this in the future, but don't have the bandwidth to think about re-opening it right now. (Just use --no-deps seems a fairer division of labour.)

[oTree-org]

How about we have just 1 channels package, but allow users to set an env var that skips installation of Daphne. Channels would check for it in setup.py:

install_requires=[
    'Django>=2.2',
    'asgiref>=3.2.10,<4',
]

if not os.getenv('CHANNELS_SLIM_INSTALL'):
    install_requires.append('daphne>=3.0,<4')

setup(
    name='channels',
    packages=find_packages(exclude=['tests']),
    install_requires=install_requires,
    # etc...
)

(The env var could also be called CHANNELS_INSTALL_WITHOUT_DAPHNE etc.)

Also in apps.py, we would wrap the daphne import in a try/except.

[oTree-org]

Here is a diff with the changes I propose: 3.0.2...oTree-org:master

[oTree-org]

Hi, any feedback on my above approach and linked PR? To summarize:

• It addresses the use case of (1) people wanting a slimmer install for their own projects, and (2) package authors who want to use Channels as a dependency but don't want the extra weight of all its dependencies, especially Twisted.
• It doesn't affect the "first time" user experience
• Installing different packages based on env vars is a commonly used technique, for example: https://github.com/certbot/certbot/blob/master/certbot-dns-google/setup.py

[carltongibson]

Hey @oTree-org. I haven't had bandwidth to look at it properly yet, but it does look like it resolves the concerns at first pass yes.

[oTree-org]

OK thank you :)

[andersk]

@oTree-org That doesn’t work. pip resolves requirements not by running setup.py for every package, but by reading a metadata file (channels.egg-info/requires.txt) included in the PyPI upload. Any conditionals need to be expressed there.

[riconnon]

I would recommend setting daphne as an extras_require perhaps

[Atem18]

@andrewgodwin Why not just do like uvicorn with uvicorn and uvicorn[standard].

That way, if people are installing uvicorn[standard], they will have a standard install. And people installing just uvicorn will just have uvicorn.

Their pull request for info : encode/uvicorn#666

[vanschelven]

Why not just do like uvicorn with uvicorn and uvicorn[standard].

Andrew already answered this one at the top of the thread

[johnthagen]

The issue I see with the --no-deps workaround is that it doesn't work for production applications that want to pin all of their dependencies with a tool such as pip-tools.

• Allow to skip dependencies (--no-deps) for individual requirements jazzband/pip-tools#817

A pip install channels[daphne] solution would solve this. I think this will become a bigger and bigger issue given the popularity of uvicorn (as it's also used in other frameworks, such as FastAPI) and how easy uvicorn slots into Gunicorn.

[collinanderson]

I personally like the idea of a separate channels-light or channels-base or channels-core package, that's basically the same thing except it doesn't have the daphne requirement. Ideally the ecosystem would allow for a [base] or [light] modifier, but until then it seems to me a separate pypi package would be the most reliable (though maybe that's a lot of work?). It would be nice to have something that's compatible with poetry add. (Either that or use channels[daphne] and make sure it's in all of the installation instructions.)

[carltongibson]

I think we could do a channels-core, that channels depended on, with a note about it in the installation docs and tutorial. (The key point being that pip install channels gives folks just getting started a working environment.)

Could is the operative work: it needs input.

• It's on my list to move the packaging to use Flit. (I don't want to use Poetry.)
• Flit takes a --ini-file option to specify the config file to build. A PoC for the dual-package layout would be a good milestone to making a decision to move forward here.

[Atem18]

Psycopg2 had to split between two packages so yes it could be the way to go.

But in that case, that means maintaining two pyproject.toml. Is it ok ?

[carltongibson]

Hi @Atem18

But in that case, that means maintaining two pyproject.toml. Is it ok ?

I don't see another way — but it's just (cough) the extra toml file, no? 🤔 — It would have to be pyproject-core.toml or something... I guess.

If someone wants to lay it out, with checklists for building etc. I'm very happy to have a look. (Otherwise I will eventually get to it, but it's not high on my personal priority list.)

[Atem18]

Hi @carltongibson, I would say so.

I think the steps are first to convert the setup.py to a working pyproject.toml, then create the pyproject-core.toml ?

[carltongibson]

@Atem18 Something like that yes... — I'd convert to using Flit, and then add the extra configuration.

[Atem18]

@carltongibson Here you go : #1840

I tested the build and install and it's working. I cannot test the publish as you can imagine but you probably can on TestPyPi :)

When the migration is done, I think we could just do a copy paste and that's all.

[johnthagen]

@carltongibson Do you think some kind of optional daphne dependency could be part of the 4.0 release? A major version bump might be a good time to make such a transition.

• Planning Channels v4.0 #1898

[collinanderson]

@johnthagen "Planning Channels v4.0" #1898 currently says "A move of the runserver command to Daphne, so that folks using other ASGI servers don't have that import overhead to use Channels."

Update: Ohh I see runserver command was already deleted in #1897, but the daphne requirement is still currently there.

[carltongibson]

@johnthagen my plan is a daphne optional install yes.

I'll settle it whilst reviewing the tutorial and docs next.

Discussion probably better over there than here 😜 ...

... but v4.0 Will resolve the underlying request here I hope.

So... 💃

[carltongibson]

@collinanderson I have this under "...then I'll look at updating the packaging..." (on #1898)

If you both (and anyone else) want to follow along there, that would be super! I'm guaranteed to miss things without extra 👀

[Atem18]

Maybe we could use #1840 ? ^^

[carltongibson]

@Atem18 it's on my list to review yep, along with #1394, and whether Hatch is worth taking up.