Upgrade to Wagtail 6.3 #478

Stormheg · 2024-10-30T20:26:58Z

Wagtail 6.3.0
Django 5.0.9
Willow 1.9.0
Pillow 11.0
Whitenoise 6.8 (nice performance improvements in this release to collectstatic!)
Removal of the unused alternative_text field, use the new description field from Wagtail 6.3 instead.

Stormheg · 2024-10-30T21:35:50Z

We worked on this branch live during Wagtail office hours tonight.

@vossisboss tried this branch with a copy of the database but we are seeing some errors with the image block changes. It's likely this has something to do with the django-pattern-library package we are using however.

If not, we should get a bug report going...

Stormheg · 2024-10-31T14:50:47Z

@vossisboss @cnk I have raised wagtail/wagtail#12514 for the ImageBlock crash we were seeing yesterday.

Stormheg · 2024-11-01T14:53:17Z

I've amended my commits to use the 6.3.0 final Wagtail release

vossisboss · 2024-11-01T15:51:22Z

I'll pull and review this some time after lunch @Stormheg

vossisboss · 2024-11-01T18:38:53Z

Hrm. Something's up here @Stormheg. Let me see if I can figure out what's going on...

Looks like the NoneType error is still there according to my local build.

laymonage · 2024-11-01T18:51:49Z

Interesting, that looks similar to wagtail/wagtail#12514 that was fixed in wagtail/wagtail#12517. Maybe there's a different case that's not handled yet? I wonder what the traceback says.

vossisboss · 2024-11-01T20:33:52Z

@laymonage Here's the trace from my local copy. I'm checking Sentry to see if the staging one is any different. I've confirmed that Wagtail 6.3 is definitely installed.

Environment:


Request Method: GET
Request URL: http://localhost:8000/

Django Version: 5.0.9
Python Version: 3.12.7
Installed Applications:
['scout_apm.django',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.sitemaps',
 'whitenoise.runserver_nostatic',
 'django.contrib.staticfiles',
 'taggit',
 'modelcluster',
 'rest_framework',
 'manifest_loader',
 'wagtail',
 'wagtail.admin',
 'wagtail.documents',
 'wagtail.snippets',
 'wagtail.users',
 'wagtail.sites',
 'wagtail.images',
 'wagtail.embeds',
 'wagtail.search',
 'wagtail.contrib.redirects',
 'wagtail.contrib.forms',
 'wagtail.api.v2',
 'wagtail.contrib.settings',
 'wagtail.contrib.typed_table_block',
 'wagtail.contrib.search_promotions',
 'wagtailio.utils',
 'wagtailio.core',
 'wagtailio.images',
 'wagtailio.standardpage',
 'wagtailio.taxonomy',
 'wagtailio.search',
 'wagtailio.navigation',
 'wagtailio.newsletter',
 'wagtailio.blog',
 'wagtailio.features',
 'wagtailio.packages',
 'wagtailio.roadmap',
 'wagtailio.services',
 'wagtailio.showcase',
 'wagtailio.areweheadlessyet',
 'wagtailio.sitewide_alert',
 'wagtailmedia',
 'pattern_library',
 'wagtailio.project_styleguide.apps.ProjectStyleguideConfig',
 'wagtailfontawesomesvg']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django_permissions_policy.PermissionsPolicyMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'wagtail.contrib.redirects.middleware.RedirectMiddleware']


Template error:
In template /app/wagtailio/project_styleguide/templates/patterns/components/streamfields/home_page_story_block.html, error at line 3
   argument of type 'NoneType' is not iterable
   1 : {% load wagtailcore_tags wagtailimages_tags %}
   2 : 
   3 :  {% for block in value %} 
   4 :     {% if block.block_type == 'get_started_block' %}
   5 :         {{ block.value.body }}
   6 :     {% else %}
   7 :         {% include_block block %}
   8 :     {% endif %}
   9 : {% endfor %}
   10 : 

Traceback (most recent call last):
  File "/venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 220, in _get_response
    response = response.render()
               ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/response.py", line 114, in render
    self.content = self.rendered_content
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/response.py", line 92, in rendered_content
    return template.render(context, self._request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/pattern_library/loader_tags.py", line 38, in render
    return super().render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/loader_tags.py", line 159, in render
    return compiled_parent._render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/pattern_library/loader_tags.py", line 38, in render
    return super().render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/loader_tags.py", line 159, in render
    return compiled_parent._render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/loader_tags.py", line 65, in render
    result = block.nodelist.render(context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/pattern_library/monkey_utils.py", line 113, in node_render
    return original_node_render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/templatetags/wagtailcore_tags.py", line 159, in render
    output = value.render_as_block(context=new_context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/blocks/stream_block.py", line 790, in render_as_block
    return self.stream_block.render(self, context=context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/blocks/base.py", line 270, in render
    return mark_safe(render_to_string(template, new_context))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/loader.py", line 62, in render_to_string
    return template.render(context, request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/django/template/defaulttags.py", line 209, in render
    for i, item in enumerate(values):
                   ^^^^^^^^^^^^^^^^^
  File "<frozen _collections_abc>", line 1026, in __iter__
    <source code not available>
                        ^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/blocks/stream_block.py", line 680, in __getitem__
    self._prefetch_blocks(raw_value["type"])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/blocks/stream_block.py", line 716, in _prefetch_blocks
    converted_values = child_block.bulk_to_python(raw_values.values())
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/blocks/struct_block.py", line 212, in bulk_to_python
    converted_values = child_block.bulk_to_python(raw_values)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/images/blocks.py", line 168, in bulk_to_python
    struct_values = super().bulk_to_python(values)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/wagtail/blocks/struct_block.py", line 208, in bulk_to_python
    if name in val:
       ^^^^^^^^^^^

Exception Type: TypeError at /
Exception Value: argument of type 'NoneType' is not iterable

vossisboss · 2024-11-01T20:38:52Z

Sentry provided some more information. I don't think you have a Sentry account @laymonage? I would send you that trace too otherwise.

cnk · 2024-11-03T06:14:19Z

This appears to be an issue specific to the Wagtail.org homepage (at least in my cursory testing). I tried reverting ImageBlocks to ImageChooserBlocks and the problem block appears to be TeaserBlock from core/blocks.py. I didn't get any further but reverting image_for_external_link = ImageBlock(required=False) to ImageChooserBlock(required=False) allows me to see the home page in my dev environment. There is a custom clean method for the block but commenting it out didn't change whether or not the homepage renders when using ImageBlock

wagtailio/images/migrations/0015_wagtailioimage_description.py

Stormheg · 2024-11-13T22:52:56Z

We opened wagtail/wagtail#12571 tonight that attempts to fix the argument of type 'NoneType' is not iterable crash once more.

The `alternative_text` field has been unused for a long while now. We have determined from a production database copy that there is a handful of images that has useful content in that field worth keeping.

6.8 has some pretty cool performance improvements to `collectstatic`!

Stormheg · 2024-11-14T13:43:25Z

I have added a commit that copies the content from alternative_text over to the description field and removes the now-obsolete alternative_text. We determined that alternative_text has been unused for a while but still contains useful data for some of the older images on the site. Might as well migrate that over now - cc @cnk.

The commit that migrates our use of ImageChooserBlocks to ImageBlocks has been extracted to #480 and removed from this PR. There isn't currently a gradual way of adopting contextual alt text, forcing editors to add it to existing images before they can successfully edit a page. We determined we are not ready for that.

With these changes in place, I think this can be considered ready for review and merge.

@vossisboss is there a way to restore a database backup of staging from before we deployed this branch? My removal of the ImageChooserBlock to ImageBlock commit from this branch will cause issues if you try to deploy this branch again, as I have removed the migration files. The removal of those files will make the database very unhappy.

zerolab · 2024-11-14T14:11:40Z

pyproject.toml

@@ -13,8 +13,8 @@ build-backend = "poetry.core.masonry.api"

 [tool.poetry.dependencies]
 python = "^3.12"
-django = "~5.0.6"
-wagtail = "~6.2.2"
+django = "~5.0.9"


why not

Suggested change

django = "~5.0.9"

django = "~5.1.4"

as well?

@zerolab that is blocked waiting on a new release of https://github.com/torchbox/django-pattern-library

argh, I keep forgetting about it.
one workaround is to build it locally and bundle the wheel with the repo, but that is not worth it. One for a follow up PR

vossisboss · 2024-11-14T19:02:38Z

@Stormheg, I'm not sure what the best way to roll back our staging database would be. That's a new one for me. I think my first inclination would be to check the Heroku dashboard and see if there's a backup restore option there. I need to write the newsletter and do some other stuff before I look into that.

Stormheg · 2024-11-20T22:55:00Z

We tested this on staging and it looks good. There is one minor issue with uploading profile pictures, the uploaded pictures aren't accessible and result in a 403 error from AWS S3. We lodged in ticket with the Torchbox sysadmin team for them to check it out as it is likely a misconfiguration and not an issue with this upgrade.

Stormheg marked this pull request as draft October 30, 2024 20:27

Stormheg mentioned this pull request Oct 31, 2024

Migrating from ImageChooserBlock to ImageBlock fails wagtail/wagtail#12514

Closed

Stormheg added 3 commits November 1, 2024 15:49

Upgrade to Wagtail 6.3

fceaf3b

Update to Django 5.0.9

4c0c04f

Add description migration

21f6e94

Stormheg force-pushed the chore/wagtail-63 branch from 05f4f2f to 2bac61d Compare November 1, 2024 14:51

Stormheg marked this pull request as ready for review November 1, 2024 14:53

vossisboss temporarily deployed to wagtail-org-staging November 1, 2024 17:29 Inactive

cnk reviewed Nov 6, 2024

View reviewed changes

wagtailio/images/migrations/0015_wagtailioimage_description.py Outdated Show resolved Hide resolved

Stormheg added 3 commits November 14, 2024 14:24

Copy Image alternative_text content to description and drop column

2ad0438

The `alternative_text` field has been unused for a long while now. We have determined from a production database copy that there is a handful of images that has useful content in that field worth keeping.

Bump Willow 1.9.0 and Pillow 11.0.0

b2228fe

Bump Whitenoise to 6.8

02ce66c

6.8 has some pretty cool performance improvements to `collectstatic`!

Stormheg force-pushed the chore/wagtail-63 branch from adc40d2 to 02ce66c Compare November 14, 2024 13:25

Stormheg mentioned this pull request Nov 14, 2024

Migrate some ImageChooserBlock instances to ImageBlock #480

Draft

Stormheg changed the title ~~Wagtail 6.3 trial~~ Upgrade to Wagtail 6.3 Nov 14, 2024

Stormheg requested review from cnk and vossisboss November 14, 2024 13:44

zerolab reviewed Nov 14, 2024

View reviewed changes

Bump wagtail to 6.3.1

7d276ee

vossisboss deployed to wagtail-org-staging November 20, 2024 21:24 View deployment

vossisboss merged commit 9f3a1fc into main Nov 27, 2024
3 checks passed

vossisboss deleted the chore/wagtail-63 branch November 27, 2024 20:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to Wagtail 6.3 #478

Upgrade to Wagtail 6.3 #478

Stormheg commented Oct 30, 2024 •

edited

Loading

Stormheg commented Oct 30, 2024

Stormheg commented Oct 31, 2024

Stormheg commented Nov 1, 2024

vossisboss commented Nov 1, 2024

vossisboss commented Nov 1, 2024

laymonage commented Nov 1, 2024

vossisboss commented Nov 1, 2024

vossisboss commented Nov 1, 2024

cnk commented Nov 3, 2024

Stormheg commented Nov 13, 2024

Stormheg commented Nov 14, 2024

zerolab Nov 14, 2024 •

edited

Loading

Stormheg Nov 14, 2024 •

edited

Loading

zerolab Nov 14, 2024

vossisboss commented Nov 14, 2024

Stormheg commented Nov 20, 2024

Upgrade to Wagtail 6.3 #478

Upgrade to Wagtail 6.3 #478

Conversation

Stormheg commented Oct 30, 2024 • edited Loading

Stormheg commented Oct 30, 2024

Stormheg commented Oct 31, 2024

Stormheg commented Nov 1, 2024

vossisboss commented Nov 1, 2024

vossisboss commented Nov 1, 2024

laymonage commented Nov 1, 2024

vossisboss commented Nov 1, 2024

vossisboss commented Nov 1, 2024

cnk commented Nov 3, 2024

Stormheg commented Nov 13, 2024

Stormheg commented Nov 14, 2024

zerolab Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Stormheg Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

zerolab Nov 14, 2024

Choose a reason for hiding this comment

vossisboss commented Nov 14, 2024

Stormheg commented Nov 20, 2024

Stormheg commented Oct 30, 2024 •

edited

Loading

zerolab Nov 14, 2024 •

edited

Loading

Stormheg Nov 14, 2024 •

edited

Loading