Add latest/stable rules to version automation

[agjohnson]

🆕 I am proposing to split this into two phases. This issue originally discussed moving implicit rules to explicit rules. Instead, I think we could start just by surfacing control that can override our implicit rules. My primary want here is user control of stable. The intent is to reduce the work needed to achieve stable automation rules.

Work we started in #4001 and #5318 will be expanded on here. We will refactor latest/stable alias management and make this more explicit to users. Users will be able to manage these aliases and our implicit rules will be surfaced in the UI for user management.

Example of psuedo rule logic we'll support with this:

• On [Most recent commit], [Set alias latest]
• On [Most recent tag/branch], [Set alias stable]

Work

• Support stable as a target for automation rules
• Some method to override/disable implicit stable version logic

Front conversations

[stsewd]

I just realized that we have some kind of automation for stable right now.

• If stable is active
• A new version is created and it's a new stable (greater than the previous one)
• If the previous version and stable were active
• The new version is activated too

https://github.com/rtfd/readthedocs.org/blob/78c34c904b347110b2cd545b4b5a80ed526590f7/readthedocs/restapi/views/model_views.py#L229-L237

[agjohnson]

One case where we might need explicit control over stable manipulation is when the project's versioning scheme doesn't quite match semver -- for instance when versions have a v prefix, like v1.2.3. These versions seem to evade our detection of stable from what I can tell.

Maybe a few options here:

• We surface the implicit stable/latest rules as default automation rules for a project. This is a lot of work, but I do like the UX here
• We let the user override our implicit rule by specifying their own stable/latest rules. We don't (yet?) try to backport this to all projects like the previous option. If a project has a stable rule, maybe we short circuit our logic and avoid implicit stable/latest version rlues
• We put special case on our versioning logic to make some of these use cases magically work. The v prefix is super common, so this could be an okay temporary solution for at least that use case.
• Could we add a feature to point stable to the most recently added/activated version?

I suppose I vote the options that turn these implicit rules, or move us closer to turning these implicit rules, into explicit automation rules.

Any other options? @stsewd any opinions on direction here?

[agjohnson]

Hrm plot twist. I set up a test scenario for the case above and it functioned like expected. Incrementing a version v1.2 to v1.3 tag did in fact update stable, unlike the user report that is noticing the opposite. There might be more to the user report or something else, like other active versions, interfering here.

It in fact was enough to enable a tagged version, and add an automation rule activating further v\d.\d tags.

[stsewd]

Think what you are mentioning here was from the bug we had with stable not being updates bc a query was killing the python process.

[agjohnson]

Yup! That was the underlying cause to the issue, so I think your point about stable already having automation still stands.

What do you think about surfacing the latest/stable rules as mock automation rules in the new UI? I'm describing hardcoded, immutable objects. Maybe this is a baby step to offering the rules as configurable automation rules, but at least gives all of the various version logic rules illustrated in one place?

[stsewd]

+1 on making that configurable, that would solve the problem from #8015 (comment)

There is a todo there

readthedocs.org/readthedocs/builds/tasks.py

Lines 324 to 326 in b523a78

	# TODO: move this to an automation rule
	promoted_version = project.update_stable_version()
	new_stable = project.get_stable_version()

I haven't thought about the implementation

[astrojuanlu]

I just found #5319 (comment) while working on #8473. Would be nice to make this an automation rule.

[helderco]

I have a monorepo and the detected stable version points to another component. The python library in my RTD is released under sdk/python/v<semver> git tags.

Pointing stable to the latest activated version would fix it, no matter if it's semver or not. I'm using an automated rule with custom match to activate the versions, waiting on #5318 to remove that prefix.

[agjohnson]

I was just reminded of this issue and I still think it would be a great UX improvement and could help us solve some issues stemming from the application being not super transparent with how we determine latest/stable.

Perhaps related to this but more likely a separate feature, it would be great to also surface a Version.is_outdated so that the user can control when we notify the reader that they should use a more up to date version.

[pspacek]

Yes please 🙏

As for surfacing Version.is_outdated, that's an excellent idea. E.g. in our project we maintain two different branches numbered like 9.16.z and 9.18.z in parallel. To make the warning accurate we would like to suppress the warning on latest 9.16.43 tag (and label all 9.16.z!=43, 9.17.z, and 9.18.z!=latest) as outdated.

[humitos]

To make the warning accurate we would like to suppress the warning on latest 9.16.43 tag (and label all 9.16.z!=43, 9.17.z, and 9.18.z!=latest) as outdated.

How do think it would be a good UX to communicate this to Read the Docs in a simple way that most users can understand?

[pspacek]

I think the current warning along the lines ~ "you are using outdated versions" is fine. It should be up to the individual projects to decide what is "outdated".

[astrojuanlu]

Would turning latest and stable into explicit automation rules that can be tweaked by users help with things like #10674 or help monorepos figure out a versioning method that works for them (given that the rules for stable assume 1 repo = 1 project)?

cc @maxschulz-COL

[maxschulz-COL]

Would turning latest and stable into explicit automation rules that can be tweaked by users help with things like #10674 or help monorepos figure out a versioning method that works for them (given that the rules for stable assume 1 repo = 1 project)?

cc @maxschulz-COL

I can only second the above! This would help us a lot!

[maxschulz-COL]

Hey lovely people - can report that on Vizro, we have come back to this, and we would still be excited for this feature :)

[humitos]

At this point, I understand that this issue and #11183 are exactly the same: "we want users to be able to define their own algorithm¹ to define the stable version via an automated rule".

I'm 💯 on that and I think it will be a really good feature 👍🏼

My only concern is about the amount of work it requires and all the parts of the code it touches; which I think it will lead to unexpected issues for some time. I've seen that @agjohnson asked about splitting this work in different chunks to try reducing the amount of work and make it in different steps, but I'm not being able to visualize how that split will reduce the work required.

Footnotes

1. we already have these algorithms implemented for the flyout, even an option to provide a custom one -- which is great! ↩

[Jackenmen]

At this point, I understand that this issue and #11183 are exactly the same: "we want users to be able to define their own algorithm1 to define the stable version via an automated rule".

This description doesn't seem to include the ability to set an automation for the most recent commit on a selected branch to trigger "stable" alias update. This was mentioned in this issue's description and, as I understood it, means I could do something that pushing a branch named "stable" does right now but without forcing you to use that exact name for the branch.

[agjohnson]

We discussed this issue today and we are going to continue with the option to add a setting that dictates if our implicit logic is used, and allow for automation of this setting with an automation rule.

• We are going to start with adding explicit control of the stable version alias first, we won't do anything with the latest alias.
• We are going to add a project field for which branch to use as the stable branch
  • This will mimic pushing to a stable branch
  • If left to its default value, the project will use our implicit logic for stable version detection
• An automation rule will then be added to automate control of this value, allowing projects to update the stable version on branch/tag creation

We do have some questions to figure out around how and when to execute the automation rules, as only running this automation rule on new branch/tag creation won't be enough. This is related to creating a stable version on project creation as well, which we discussed removing.

Long term, we'd aim to move the implicit stable version logic to an explicit rule, but this logic does have conditions that we don't have in automation rules.

We'll know more as we start breaking up this code, but this does not feel like a huge amount of work at the moment.

@Jackenmen I believe the way we would implement this would cover what you're describing

[Jackenmen]

Should be fine, thank you for including that use case!

Have you thought about how this feature would work when a branch gets deleted at some point, i.e. does it switch the alias to whatever is the latest tagged version (until the branch potentially gets created layer) or does it not do anything until the next (newer) tagged version is created (or until the branch is recreated)?
Both seem like options with some upsides and probably ways to workaround their limitations so am mostly just curious, if it is something you have any thoughts on.

[humitos]

Have you thought about how this feature would work when a branch gets deleted at some point, i.e. does it switch the alias to whatever is the latest tagged version (until the branch potentially gets created layer) or does it not do anything until the next (newer) tagged version is created (or until the branch is recreated)?

One of the notes we have from our meeting is "If users set the stable version manually (for example to their existing rel/stable branch or similar), it will never get updated"; so I think the answer to your question is "it won't do anything" 😄

[maxschulz-COL]

Hey guys, happy to see that this gets traction 😃 . Am I right in understanding that the aim is now to define stable by commits to specific branch?

Would it also consider the case where we define a tag manually? For example -v0.1.12 or smth?

[stsewd]

@maxschulz-COL stable would point to a branch (which will use the latest commit that was built), or a tag.

[Jackenmen]

Disclaimer: I'll be asking some dumb questions in perhaps a bit argumentative way but I'm just trying to understand the differences between different parts that are proposed, I have no ill will 😄 At the end, I started just throwing random thoughts without concrete questions so I don't really know what I expect to gain from it :P

Have you thought about how this feature would work when a branch gets deleted at some point, i.e. does it switch the alias to whatever is the latest tagged version (until the branch potentially gets created layer) or does it not do anything until the next (newer) tagged version is created (or until the branch is recreated)?

If users set the stable version manually (for example to their existing rel/stable branch or similar), it will never get updated

I don't really know what part of the feature this is referring to exactly. What does setting manually here mean?

---

Based on Anthony's message, there's:

• We are going to start with adding explicit control of the stable version alias first, we won't do anything with the latest alias.

which perhaps actually means you could completely manually set the stable alias to point at a specific build but the other two points definitely have some degree of automation (since they update on push) so they're probably not manual?

i.e. the second point:

• We are going to add a project field for which branch to use as the stable branch
  • This will mimic pushing to a stable branch

seems to mean setting a fixed branch name whose updates will trigger alias updates.

But then we also have the third point:

• An automation rule will then be added to automate control of this value, allowing projects to update the stable version on branch/tag creation

which, if it were set to "update the stable version on branch creation" seemingly means the same thing but it says it's an "automation rule" so maybe it offers more features over the second point? Does the second point offer something that this point doesn't?

---

I imagine I would want to use a second or third point, both of which perform some automation so I'm not entirely sure what your answer means in the context of those points.

Concretely, would it be possible to have both "on branch update" and "on tag update" rules enabled at the same time? Would they do something sensible? My previous question presented a situation where, if a stable-docs branch doesn't exist, I was thinking that something should either happen on the removal or something should happen when a new tag is pushed, not that nothing ever happens again (unless the branch is created again or rules get changed).
I believe that the "something should happen when a new tag is pushed, if stable-docs branch doesn't exist" is the situation we have now but since you said that the stable alias would never get updated, it is no longer clear to me that this behaviour would remain.

I managed to confuse myself with this last part because I only look at the case of the stable-docs branch doesn't exist as I made another assumption that branch rule would be more important than the tag rule but maybe they could live in harmony 😄 Or maybe everything I said is irrelevant because you'd be able to only set one rule at a time. The problem I see with that variant is that it seems like it would require manually going to RTD project settings to change rules each time you decide on switching from branch-based stable to tag-based stable which would be a bit of a downgrade compared to RTD we have now by being more manual. But maybe it's not worth keeping the current behaviour in any way...

[maxschulz-COL]

@maxschulz-COL stable would point to a branch (which will use the latest commit that was built), or a tag.

And that tag could be customizeable / "user defined algrothm" --> I am thinking of mono-repos with one main branch but two packages with two sets of documentation

[stsewd]

Hopefully this clarifies the feature:

• We will allow users to manually point the stable version to a branch or tag, when the branch or tag receives an update, the stable version will also follow those updates. If this setting is used, the stable version will be static, this is, it won't change automatically if a new tag or branch is pushed.
• In order to allow users to change the stable version automatically based on some rules/patterns, we will introduce an automation rule to allow that. More complex behaviors could be achieved by using our API.
• If the tag or branch the stable version points to is deleted, the stable version won't be deleted (we don't delete content of active versions automatically, only when they are explicitly deactivated). Automation rules can also run when a branch/tag is deleted, so that same rule that is used for updating the stable version when a new branch/tag is created could be used for when a tag/branch is deleted.

[astrojuanlu]

For reference (in case someone missed that context), Automation Rules is a concept that already exists on RTD https://docs.readthedocs.io/en/stable/automation-rules.html