Update UI message feed via polling

[cc-a]

Description

This PR makes the Kafka message feed auto-update. It:

• Creates a separate partial template for the contents of the message feed.
• Adds a dedicated partial view function to render message data into the template.
• Updates the process_manager index page template to include a htmx element that polls the messages view and populates new messages into the page.

Fixes #121

Type of change

• Documentation (non-breaking change that adds or improves the documentation)
• New feature (non-breaking change which adds functionality)
• Optimization (non-breaking, back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)
• Breaking change (whatever its nature)

Key checklist

• All tests pass (eg. python -m pytest)
• The documentation builds and looks OK (eg. python -m sphinx -b html docs docs/build)
• Pre-commit hooks run successfully (eg. pre-commit run --all-files)

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added or an issue has been opened to tackle that in the future. (Indicate issue here: # (issue))

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.39%. Comparing base (e231526) to head (057dd3e).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #141      +/-   ##
==========================================
+ Coverage   79.25%   79.39%   +0.14%     
==========================================
  Files          25       25              
  Lines         294      296       +2     
==========================================
+ Hits          233      235       +2     
  Misses         61       61              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dalonsoa]

I'm not able to make this work. I consistently get:

Internal Server Error: /process_manager/partials/messages/
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py", line 350, in execute
    return super().execute(query)
           ^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/decorators.py", line 60, in _view_wrapper
    return view_func(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/process_manager/views/partials.py", line 61, in messages
    with transaction.atomic():
  File "/usr/local/lib/python3.11/site-packages/django/db/transaction.py", line 216, in __enter__
    connection.set_autocommit(
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 482, in set_autocommit
    self._start_transaction_under_autocommit()
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py", line 326, in _start_transaction_under_autocommit
    self.cursor().execute(f"BEGIN {self.transaction_mode}")
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 122, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py", line 350, in execute
    return super().execute(query)
           ^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.OperationalError: database is locked
[09/Oct/2024 12:13:19] "GET /process_manager/partials/messages/ HTTP/1.1" 500 117543
[09/Oct/2024 12:13:20] "POST /process_manager/partials/process_table/ HTTP/1.1" 200 10754

I'm not sure what might be going wrong.

[dalonsoa]

Now is working well. No idea what the problem was.

[jamesturner246]

Just a comment about instant loading the messages. Else LGTM 👍

[jamesturner246]

Maybe it's worth substituting a repeat delay hx-get into the bootstrap instant load hx-get in index, as you've done for the table?