Running ChannelsLiveServerTestCase does not create database for browser

[alfonsrv]

Using ChannelsLiveServerTestCase as described in documentation's Tutorial Part 4: Automated Testing yields the yellow error page connection to server at "127.0.0.1", port 5432 failed: FATAL: database "test" does not exist. Running database-altering code within a test function works and the database is available, but the frontend worker serving the browser request (using Daphne) seems to be totally detached from that database. Somehow the asgiref workers are working in another thread?

I created a small test repository with a small readme to reproduce the issue: https://github.com/alfonsrv/django-asgitest

Tried using macOS Ventura + Ubuntu 22.04 with Chrome.

[carltongibson]

Are you sure your environment variables are being set correctly? (I'd expect the test DB name to be test_testasgi from the sample .env file + settings.py you have in your repo.)

[alfonsrv]

Pardon me, coming from pytest I was unaware unittest already required a database and then simply cloned the already-existing one, prefixing it with test_. While the database error is gone, data created within a test function still cannot be used from the browser.

E.g. using the following code does not allow the user to log into the admin using the browser:

user_staff = User.objects.create(
    username='test',
    email='[email protected]',
    password=make_password('hunter42'),
    is_staff=True,
    is_superuser=True
)
self.client.login(
    username=user_staff.username, 
    password='hunter42'
)
# returns True

self.driver.get(self.live_server_url + "/admin/")
self.driver.find_element('name', 'username').send_keys(user_staff.username)
self.driver.find_element('name', 'password').send_keys('hunter42')
self.driver.find_element(By.CSS_SELECTOR, "input[type='submit']").click()
# error in admin - login does not work

[carltongibson]

Ok, but in general that works, so you need to put a breakpoint there and see why the login isn't working. (This isn't really a channels issue.)

[alfonsrv]

It's not a channels issue per-se, but what I assume is an asgiref issue / ChannelsLiveServerTestCase issue, since the intended use to test the frontend does not work as outlined in the docs.

It's like the database is out of sync between the sync test routing and the async server worker thread. I threw in a pdb in the test function, but everything there is working fine there. Not sure how I would debug the server function serving the frontend.

[carltongibson]

It works if you follow the steps in the tutorial. I'm not sure what else I can say given the info provided. Sorry.

[alfonsrv]

I just followed the steps in the tutorial and setting up a test database that is available via the frontend does not work as expected.

Users that are created as part of the regular Python functions are not available to the frontend (see chat.tests.DatabaseTests). The only reason the tutorial "works" is because it does not use any part of the ORM but unauthenticated views testing the mere websockets functionality.

One could argue that this is a sufficient for unit tests and the rest if out of scope for this library. However I would argue if you provide a dedicated test server for the use case, it should work with the ORM, because it's unlikely that it's used only with unauthenticated interactions.

[carltongibson]

OK, your test project has way too much going on to identify any issue. Can you trim it down to a minimal please, with a clear error case, rather than a sleep. (The initial ticket here was "Database not created", so if you can clarify that too, that would be great.

I'd move the user creation into setup, and then inspect the DB to make sure that's actually committed. Then compare Django's Admin Selenium test cases, which do this at length.

[alfonsrv]

OK, your test project has way too much going on to identify any issue

Not sure what exactly you are referring to. I simply implemented the tutorial boilerplate in chat and config-related settings as requested.

I slimmed down chat.tests – including a SyncDatabaseTests and AsyncDatabaseTests, both testing for the same result, only using the different servers. The former one works.

then inspect the DB to make sure that's actually committed. Then compare Django's Admin Selenium test cases, which do this at length

From what I can see in e.g. the Django admin_changelist test, the user is simply created, then used to log in thereafter. Nevertheless I added a simple check to test_user_model_creation that asserts the user is present before opening the browser.

[carltongibson]

Ok, thanks. I will try and have a look, and perhaps expand the tutorial example to show interactions with the DB.

[alfonsrv]

Heya! Have you had a chance to look into it?

[carltongibson]

Hi @alfonsrv — no, not as yet. It's on my list for later in the autumn, but short of someone (you maybe?) digging deeper (and highlighting either the error in the project or the bug in the test case) it'll have to wait until I can get to it.

[ashtonrobinson]

#2033

[alfonsrv]

@ashtonrobinson I tried using your PR but it leads to a ProgrammingError at /chat/test/ yellow error page, as if migrations didn't run for the frontend / test database.

But it seems like you're onto something – because using the async test case (chat.tests.AsyncDatabaseTests) I created, it appears to use the default database instead of the test database.

And users that are created as part of python manage.py createsuperuser are listed in the AsyncDatabaseTests frontend, while they are not when using the regular unit test SyncDatabaseTests.

[carltongibson]

From the PR...

a multiprocessing.Process object which takes care of creating a new process. This uses the SpawnProcess of multiprocessing class on Windows and macOS. Linux uses the Fork method...

Ah, spawn vs fork

This will be related to the discover runner updates in Django 4.1. https://docs.djangoproject.com/en/5.0/releases/4.1/#tests

See https://code.djangoproject.com/ticket/31169 — We'll likely need to use some of the added test setup steps.

Progress. 🎁

[carltongibson]

Just to add in the meantime, if the spawn startup is the issue running the tests with --parallel=1 should function as a workaround.

[alfonsrv]

That doesn't work either.

How can something as fundamental as tests (albeit in the context of database interactions) be kept broken for what is essentially a core Django library for this long?

[carltongibson]

@alfonsrv by it not being exercised. Django is built on volunteer effort, so short of someone picking it up and investigating it can take a while.

Since you're using this, perhaps you could be the person to work out the cause and resolve it.

[alfonsrv]

I tried looking into it, but I'm afraid it's a bit too advanced for my understanding.

Willing to sponsor and offer a bounty though.

[carltongibson]

@alfonsrv the first step would be a failing test case demonstrating the issue. If folks can instantly get to the problem it's much easier to work on.

[alfonsrv]

That exists: https://github.com/alfonsrv/django-asgitest – the test cases inherit the same base class, to run the same code both Sync and Async.

Mentioned the repo here previously. It has some boilerplate code due to following the channels tutorial – as I was asked to previously – but the test cases are very basic.

[carltongibson]

I really meant as a unit test against the channels test suite, but yes.

The bottom line is that until somebody can find the time to dig into it, this will sit there. It's on my list, but I have quite a few 🤹 other things which are more pressing to me, so it's not something I'm going to get to quickly.

[alfonsrv]

Any update?

[bigfootjon]

@alfonsrv: can you provide the output of what's failing in your example repo? (i.e. the stack trace)

[alfonsrv]

There's no relevant stack trace – the test fails when attempting to look up a user in the frontend via browser that was previously created by backend code.

However I finally managed to fix this issue using pytest by jumping through some hoops.

[Checkm3out]

Hello, Iv also run into this issue,

During testing, i can see that my DB is being created and migrations are being added.
However, when connecting with selenium to daphne, daphne seems to be using the actual database connection defined in the settings - rather than the modified settings created by TransactionTestCase

To reproduce:

Check your database name within the test

   for db_name, db_config in settings.DATABASES.items():
        print(f"db_config: {db_config}")

and then also in your view called by selenium

   for db_name, db_config in settings.DATABASES.items():
        print(f"db_config: {db_config}")

You will see that the output inside your test will show
{'NAME': 'test_defaultdatabase' ...

and the output in your daphne view will be
{'NAME': 'defaultdatabase' ...

Whereas if you use StaticLiveServerTestCase instead of ChannelsLiveServerTestCase

Both outputs will show the test_defaultdatabase
..
I will attempt keep digging for a possible solution, but hopefully this helps narrow down the issue

I would rename the issue to - "ChannelsLiveServerTestCase Daphne server does not read from the Mirrored Test database's created by django" based on this information

[Checkm3out]

Quick solution, but can probably be better implemented -

In ChannelsLiveServerTestCase
Change ProtocolServerProcess = CustomDaphneProcess

Override the run function in DaphneProcess to modify your database config

class CustomDaphneProcess(DaphneProcess):
#We just need to override the run - everything else is the same
def run(self):
# OK, now we are in a forked child process, and want to use the reactor.
# However, FreeBSD systems like MacOS do not fork the underlying Kqueue,
# which asyncio (hence asyncioreactor) is built on.
# Therefore, we should uninstall the broken reactor and install a new one.
_reinstall_reactor()

    '''
    Load Django settings and ensure the test database is used
    START Addition 1 of 2
    '''
    if not settings.configured: #Fix For raise AppRegistryNotReady("Apps aren't loaded yet.")
        django_setup()  # Ensure Django is fully set up before using settings
    '''
    END Addition 1 of 2
    '''
    
    from twisted.internet import reactor

    from daphne.endpoints import build_endpoint_description_strings
    from daphne.server import Server

    application = self.get_application()
    
    '''
    Start Addition 2 of 2
    Ensure the database names starts with 'test_'
    '''
   if not settings.DATABASES[list(settings.DATABASES.keys())[0]]['NAME'].startswith('test_'):
        print(f"Database does not start with 'test_': {settings.DATABASES['default']['NAME']}")
        
        for db_name, db_settings in settings.DATABASES.items():
            db_settings['NAME'] = f"test_{db_settings['NAME']}"
        print(f"All database names modified")
    '''
    END Addition 2 of 2
    '''
    
    try:
        # Create the server class
        endpoints = build_endpoint_description_strings(host=self.host, port=0)
        self.server = Server(
            application=application,
            endpoints=endpoints,
            signal_handlers=False,
            **self.kwargs
        )
        # Set up a poller to look for the port
        reactor.callLater(0.1, self.resolve_port)
        # Run with setup/teardown
        if self.setup is not None:
            self.setup()
        try:
            self.server.run()
        finally:
            if self.teardown is not None:
                self.teardown()
    except BaseException as e:
        # Put the error on our queue so the parent gets it
        self.errors.put((e, traceback.format_exc()))

This leads me to a new issue, that the tests override_settings are not applied to the daphne server, And i have not figured out how to apply patches to the selenium views yet

Update: To add patches you can also do that in the Run before you create the server class - and you can change system settings the same way you change the database names

[carltongibson]

daphne seems to be using the actual database connection defined in the settings - rather than the modified settings created by TransactionTestCase

Thanks @Checkm3out — that gives us a lead. There have been various similar issues over the years on Django that may well be ≈the same. 🕵️