feat: Add rate limit handling for GitHub client

[lukaspj]

This is a recreation of #4926 as that one seems abandoned by the author

What

• Add retry mechanism for GitHub API secondary rate limit
• Implement a retry mechanism for GitHub client API calls when encountering secondary rate limit responses.

Why

• In large-scale deployments, a single PR can affect hundreds of projects.
• When attempting to plan/apply for a non-mergeable PR, Atlantis may try to set the PR status hundreds of times in a short period, triggering GitHub's secondary API rate limit.
• This results in Atlantis appearing to stop mid-operation: some projects are marked as failed, and no comments are posted.
• This issue can occur in other scenarios where Atlantis is used at scale.

Implementation

• Utilize the go-github-ratelimit library, as recommended in the official go-github README:

  "You can use go-github-ratelimit to handle secondary rate limit sleep-and-retry for you."

Testing

• Added a test that verifies an API request completes successfully after responding with API secondary rate limit errors multiple times.

References

go-github README
go-github-ratelimit library

[lukaspj]

For the record, we have tested this out on our setup which runs hundreds of Atlantis jobs simultaneously and it greatly improved our experience

[chenrui333]

Sounds like a good idea, @lukaspj is it configurable?

[lukaspj]

I can make it configurable for sure, the original PR had discussions on this as well. I’m indifferent towards configuration options because the current default seems very stable for us and we had rate limiting problems constantly before

[lukaspj]

I can make it configurable for sure, the original PR had discussions on this as well. I’m indifferent towards configuration options because the current default seems very stable for us and we had rate limiting problems constantly before