Use Releases instead of Tags

[ericLemanissier]

Currently, conan-center-bot uses git tags to identify new version, and their download URLs. This is not ideal because more and more recipes are using release tarballs instead of git archives (this ensures sha stability).
With the current stuation,
Github and gitlab provide API to list releases and their artifact (eg https://gitlab.freedesktop.org/api/v4/projects/wayland%2Fwayland/releases and https://api.github.com/repos/openssl/openssl/releases). Using these API when available would:

• allow more projects to be supported by CCB (eg wayland, wayland-protocols, pulseaudio)
• allow currently supported projects to use release tarballs instead of git tarballs

The main problem is that both github and gitlab have rate limits on API.
For github:

• unauthenticated consumers: 60 requests/hour (clearly not enough)
• GITHUB_TOKEN: 1000 requests/hour (could be enough, there are currently 1500 recipes on CCI, but not all come from github)
• Personal Access Token: 5000 requests/hour (should be ok in the long term)

Gitlab rate limits are configurable per instance, so it's harder to have a clearer picture of what's possible.

Is it worth that I spend some time to implement these two new types of upstream and make a PR, or do you see a blocker right away ?

[ericLemanissier]

friendly ping @qchateau what's your take on the above ?

[qchateau]

I'm fine with adding support for more ways to find releases.
IIRC I already use a GitHub token to find whether there are already open PR on CCI for a given update so there should be a way to reuse it, you don't have to do it yourself.
Just make sure that breaching the rate limits doesn't crash the whole run - breaching the rate limit of the gitlab token should only prevent the recipes that rely on it from being auto-updated.
For GitHub it's a bit more tricky because we need the token to detect open PRs, which impacts all recipes. But I suppose 1k request/hour give us some headroom

[ericLemanissier]

Ok, I should be able to see if/when/how rate-limit happens on my fork's workflow