Skip to content

Commit

Permalink
Merge pull request #2 from justinrixx/docs
Browse files Browse the repository at this point in the history
docs: document default behavior and available options
justinrixx authored Dec 18, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
2 parents 55a3ff1 + afc30ac commit e399203
Showing 4 changed files with 85 additions and 3 deletions.
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Contributing

Feel free to open an issue if you have questions, comments, concerns, or bugs. I'm happy to review PRs, but please start the conversation in issues before requesting a review. This isn't my full-time job, so I may not respond immediately, but I have gotten value from this package and want to make sure it is the best it can be to help others.
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -3,9 +3,9 @@
`retryhttp` allows you to add HTTP retries to your service or application with no refactoring at all, just a few lines of configuration where your client is instantiated. This package's goals are:

- Make adding retries easy, with no refactor required (as stated above)
- Provide a good starting point for retry behavior
- Make customizing retry behavior easy
- Allow for one-off behavior changes without needing multiple HTTP clients
- [Provide a good starting point for retry behavior](./docs/default.md)
- [Make customizing retry behavior easy](./docs/options.md)
- [Allow for one-off behavior changes without needing multiple HTTP clients](./docs/options.md#example)
- 100% Go, with no external dependencies (have a peek at `go.mod`)

## How it works
@@ -26,6 +26,13 @@ client := http.Client{
client := http.Client{
Transport: retryhttp.New(
// optional retry configurations
retryhttp.WithShouldRetryFn(func(attempt retryhttp.Attempt) bool {
return attempt.Res != nil && attempt.Res.StatusCode == http.StatusServiceUnavailable
}),
retryhttp.WithDelayFn(func(attempt retryhttp.Attempt) time.Duration {
return expBackoff(attempt.Count)
}),
retryhttp.WithMaxRetries(2),
),
// other HTTP client options
}
23 changes: 23 additions & 0 deletions docs/default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Default behaviors

The default behaviors of this package are mostly implemented in `DefaultShouldRetryFn` and `DefaultDelayFn`, which you may choose to read for yourself. A high level description of their logic follows.

## `DefaultShouldRetryFn`

- If an error occured (non-nil `attempt.Err`, nil `attempt.Res`), and if that error is a DNS error, the request is retried. This is because it never reached the target server due to failing on the DNS lookup.
- If an error occured and if that error is a common timeout error (see `IsTimeoutErr`), the request is retried only if it is guessed to be idempotent[^1].
- If no error occured an a non-nil response was returned, the request is retried if the response indicates the server expects a retry[^2].
- If the request is guessed idempotent[^1] and the status code is 502 or 503, the request is retried
- Otherwise, the request is not retried

The methods considered idempotent and the status codes considered retryable can be tweaked by using `CustomizedShouldRetryFn` instead.

## `DefaultDelayFn`

- If the `Retry-After` header is provided, a wait duration is derived from its value. This field may be a non-negative integer representing seconds, or a timestamp. Once a duration is obtained, jitter of magnitude up to one third ($\frac{1}{3}$) is added or subtracted from that duration as jitter.
- If no `Retry-After` header is provided, exponential backoff with jitter is used. The algorithm used [is described here](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) as "full jitter". The exponential base used is 250ms, and it is capped at 10s.

The jitter magnitude, exponential base, and exponential backoff cap can be tweaked by using `CustomizedDelayFn` instead.

[^1]: A request is guessed idempotent if it uses an [idempotent HTTP method](-editor.org/rfc/rfc9110.html#name-idempotent-methods) or includes the `X-Idempotency-Key` or `Idempotency-Key` header.
[^2]: A status code of 429 indicates the server did not process the request and anticipates the caller to retry after some delay. Similarly, the `Retry-After` response header indicates the request should be retried after a delay.
49 changes: 49 additions & 0 deletions docs/options.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Options

`retryhttp.Transport` can be customized with several options. In general, each option that can be specified at creation time has an equivalent helper function for overriding the option using the request `Context`. An option set on the `Context` takes precedence over an option set on the `Transport`.

| Option | Context Equivalent | Default Value | Description |
| ------ | ------------------ | ------------- | ----------- |
| `WithTransport` | none | `http.DefaultTransport` | The internal `http.RoundTripper` to use for requests. |
| `WithShouldRetryFn` | `SetShouldRetryFnOnContext` | `DefaultShouldRetryFn` | The `ShouldRetryFn` that determines if a request should be retried. `DefaultShouldRetryFn` is a good starting point. If you're only looking to make minor tweaks, `CustomizedShouldRetryFn` may be appropriate. |
| `WithDelayFn` | `SetDelayFnOnContext` | `DefaultDelayFn` | The `DelayFn` that determines how long to delay between retries. If `DefaultDelayFn` doesn't solve your use-case, `CustomizedDelayFn` may be appropriate. |
| `WithMaxRetries` | `SetMaxRetriesOnContext` | 3 | The maximum number of retries to make. Note that this is the number of _retries_ not _attempts_, so a `MaxRetries` of 3 means up to 4 total attempts: 1 initial attempt and 3 retries. Note also that if your `ShouldRetryFn` returns `false`, a retry will not be made even if `MaxRetries` has not been exhausted. |
| `WithPreventRetryWithBody` | `SetPreventRetryWithBodyOnContext` | `false` | Whether to prevent retrying requests that have a HTTP body. Any request that has any chance of needing a retry must buffer its body into memory so that it can be replayed in subsequent attempts. This may or may not be appropriate for certain use-cases, which is why this option is provided. |
| `WithAttemptTimeout` | `SetAttemptTimeoutOnContext` | No timeout | A per-attempt timeout to be used. This differs from an overall timeout in that the timeout is reset for each attempt. Without a per-attempt timeout, the overall timeout could be exhausted in a single attempt with no time left for subsequent retries. Providing `time.Duration(0)` here removes the timeout. |

## Example

```go
client := http.Client{
Transport: retryhttp.New(
retryhttp.WithShouldRetryFn(attempt retryhttp.Attempt) bool {
// only retry HTTP 418 statuses
if attempt.Res != nil && attempt.Res.StatusCode == http.StatusTeapot {
return true
}
return false
},
retryhttp.WithMaxRetries(2),
retryhttp.WithAttemptTimeout(time.Second * 10),
),
}

ctx := context.TODO
ctx = retryhttp.SetShouldRetryFnOnContext(ctx, func(attempt retryhttp.Attempt) bool {
// retry any error
if attempt.Err != nil {
return true
}
return false
})
ctx = retryhttp.SetMaxRetriesOnContext(ctx, 1) // only 1 retry
ctx = retryhttp.SetAttemptTimeoutOnContext(ctx, 0) // remove attempt timeout

req, err := http.NewRequest(http.MethodGet, "example.com", nil)
...

// add augmented context to the request: retries will abide by the overrides
// instead of the orginial configurations
res, err := client.Do(req.WithContext(ctx))
...
```

0 comments on commit e399203

Please sign in to comment.