Redirects: improvements from design doc

[stsewd]

Implements https://dev.readthedocs.io/en/latest/design/redirects.html

• New fields: status code, description, enabled, force, position.
• Redirects now have an explicit order, the code from automation rules was re-used.
  For the templates, I also re-used the styles from automation rules.
• Prefix redirects are removed in favor of exact redirects + wildcard
• API V3 and form will show an error if the user tries to use the old style wildcard form ($rest), or use the old types.
• Variables like path/full_path were renamed to filename and path, we use those names on proxito and friends.
• Redirect are not run on pull request previews.
• "Sphinx redirects" were renamed.
• get_redirect_path was previously re-running the same checks from get_redirect_path_with_status, we don't do this anymore, we avoid duplication and should be a little faster.
• The query used to match redirects is improved to run just the required redirects, depending on the path and filename. Did a quick test, and performance is kind of the same, hopefully we will see an improvement on a more large scale, but evaluation of redirects is fast currently.
• Docs were re-structured a little, now all examples are after the list of redirect types, since they apply to all types.
• The "limitations" section was at the start of the document, but they apply to user-redirects (not built-in redirects).
• Matching ext-theme changes at Redirects: match recent improvements ext-theme#237.
• Docs: https://docs--10881.org.readthedocs.build/en/10881/user-defined-redirects.html
• We need to decide what to do with a couple of redirects that are already using the new syntax, since it may be unexpected for some projects if they start working, full list at https://readthedocs.slack.com/archives/C02DNG2HSNP/p1701962946332549.

Questions

• Should we allow forced redirects now? Pages are cached on CF, and we don't have many projects using forced redirects, evaluation of redirects is fast.

How to deploy

• Deploy web extra
• Run the migration redirects 0006
• Deploy webs
• Run the data migration redirects 0007
• Run the following code from a django shell

from readthedocs.core.resolver import Resolver
from readthedocs.projects.models import Project

for project in Project.objects.filter(redirects__isnull=False).distinct():
    default_path = Resolver().resolve_path(project)

    for redirect in project.redirects.all():
        # Migrate exact redirects with a wildcard to new syntax.
        if redirect.redirect_type == "exact" and redirect.from_url.endswith(
            "$rest"
        ):
            redirect.from_url = redirect.from_url.removesuffix("$rest") + "*"
            redirect.to_url = redirect.to_url + ":splat"

        # Migrate prefix redirects to exact redirects with a wildcard.
        if redirect.redirect_type == "prefix":
            redirect.redirect_type = "exact"
            redirect.from_url = redirect.from_url + "*"
            redirect.to_url = f"{default_path}/:splat"

        # Saving will normalize from_url and to_url as needed.
        redirect.save()

We would experiment some redirects not working while the migration takes place,
we don't have that many redirects, so it should be quick. We need to save each redirect individually so they are normalized as the new code expects.

Screenshots

Closing credits?

• Fixes Forced redirect /$rest -> example.com redirects PR previews (external domain) to target domain #9614
• Closes Exact redirects: allow use of $lang and $version #9240
• Closes Document order of redirects (recently created are matched first) #8309
• Closes Refactor redirects code #6405
• Closes Validate user input URLs on redirect form #4555
• Closes Improve redirects feature #4221
• Closes Interaction between redirects and trailing slashes #10962

[stsewd]

This can be a mixin + a class attribute, not strong opinion.

[stsewd]

All this code comes from automation rules, it's the same code, just changed to use self.position_field_name instead of priority.

[ericholscher]

This looks like a great change. I'd be curious to hear more about the performance changes that come from this. If we think we can do redirect queries on each pageview, I'd be strongly in favor of removing the limitations around Forced redirects, and enabling them by default I think.

[ericholscher]

Love all the examples in this page 💯

[ericholscher]

I don't understand the /``/``/index.html -- is that just the path ///index.html?

[stsewd]

It's A/B, I have reworded this to make it less confusing.

[stsewd]

If we think we can do redirect queries on each pageview, I'd be strongly in favor of removing the limitations around Forced redirects, and enabling them by default I think.

We can decide about enabling them after shipping these changes. Currently, matching redirects takes less than 1ms, so they are already fast, I don't think we will have any problems with that.

[ericholscher]

If we think we can do redirect queries on each pageview, I'd be strongly in favor of removing the limitations around Forced redirects, and enabling them by default I think.

We can decide about enabling them after shipping these changes. Currently, matching redirects takes less than 1ms, so they are already fast, I don't think we will have any problems with that.

Sounds good. I'm 👍 on merging this and doing that work next.