Skip to content

Commit

Permalink
doc: sync circuitbreaker.md (#838)
Browse files Browse the repository at this point in the history
Co-authored-by: Yi Duan <[email protected]>
  • Loading branch information
alice-yyds and AsterDY authored Oct 23, 2023
1 parent 507de02 commit d40309a
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 54 deletions.
106 changes: 52 additions & 54 deletions content/en/docs/kitex/Tutorials/service-governance/circuitbreaker.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
---
title: "Circuit Breaker"
date: 2021-08-31
weight: 5
keywords: ["Kitex", "Circuit Breaker"]
description: "This doc covers Kitex Circuit Breaker use guide and principle introduction."
date: 2023-10-23
weight: 1
keywords: ["Kitex - EN","Circuit Breaker"]
description: ""
---

Kitex provides a default implementation of Circuit Breaker, but it's disabled by default.
# Circuit Breaker

## Introduction

Kitex provides a default implementation of Circuit Breaker, but it’s disabled by default.

The following document will introduce that how to enable circuit breaker and configure the policy.

Expand All @@ -16,41 +20,43 @@ The following document will introduce that how to enable circuit breaker and con

```go
import (
...
"github.com/cloudwego/kitex/client"
"github.com/cloudwego/kitex/pkg/circuitbreak"
"github.com/cloudwego/kitex/pkg/rpcinfo"
...
"github.com/cloudwego/kitex/client"
"github.com/cloudwego/kitex/pkg/circuitbreak"
"github.com/cloudwego/kitex/pkg/rpcinfo"
)

// GenServiceCBKeyFunc returns a key which determines the granularity of the CBSuite
func GenServiceCBKeyFunc(ri rpcinfo.RPCInfo) string {
// circuitbreak.RPCInfo2Key returns "$fromServiceName/$toServiceName/$method"
return circuitbreak.RPCInfo2Key(ri)
// circuitbreak.RPCInfo2Key returns "$fromServiceName/$toServiceName/$method"
return circuitbreak.RPCInfo2Key(ri)
}

func main() {
// build a new CBSuite with
cbs := circuitbreak.NewCBSuite(GenServiceCBKeyFunc)

var opts []client.Option
// add to the client options
opts = append(opts, client.WithCircuitBreaker(cbs))
// init client
cli, err := echoservice.NewClient(targetService, opts...)
// update circuit breaker config for a certain key (should be consistent with GenServiceCBKeyFunc)
// this can be called at any time, and will take effect for following requests
cbs.UpdateServiceCBConfig("fromServiceName/toServiceName/method", circuitbreak.CBConfig{
Enable: true,
ErrRate: 0.3, // requests will be blocked if error rate >= 30%
MinSample: 200, // this config takes effect if sampled requests are more than `MinSample`
})

// send requests with the client above
...
// build a new CBSuite with
cbs := circuitbreak.NewCBSuite(GenServiceCBKeyFunc)

var opts []client.Option
// add to the client options
opts = append(opts, client.WithCircuitBreaker(cbs))
// init client
cli, err := echoservice.NewClient(targetService, opts...)
// update circuit breaker config for a certain key (should be consistent with GenServiceCBKeyFunc)
// this can be called at any time, and will take effect for following requests
cbs.UpdateServiceCBConfig("fromServiceName/toServiceName/method", circuitbreak.CBConfig{
Enable: true,
ErrRate: 0.3, // requests will be blocked if error rate >= 30%
MinSample: 200, // this config takes effect if sampled requests are more than `MinSample`
})

// send requests with the client above
...
}


```

### Introduction
Expand All @@ -59,29 +65,24 @@ Kitex provides a set of CBSuite that encapsulates both service-level breaker and

- Service-Level Breaker

Statistics by service granularity, enabled via WithMiddleware.

The specific division of service granularity depends on the Circuit Breaker Key, which is the key for breaker statistics. When initializing the CBSuite, you need to pass it in **GenServiceCBKeyFunc**. The default key is `circuitbreak.RPCInfo2Key`, and the format of RPCInfo2Key is ` fromServiceName/toServiceName/method`.

- Statistics by service granularity, enabled via WithMiddleware.
- The specific service granularity depends on the Circuit Breaker Key, which is the key for breaker statistics. When initializing the CBSuite, you need to pass it in **GenServiceCBKeyFunc**. The default key is `circuitbreak.RPCInfo2Key`, and the format of RPCInfo2Key is `fromServiceName/toServiceName/method`.
- Instance-Level Breaker

Statistics by instance granularity, enabled via WithInstanceMW.

Instance-Level Breaker is used to solve the single-instance exception problem. If it's triggered, the framework will automatically retry the request.

Note that the premise of retry is that you need to enable breaker with **WithInstanceMW**, which will be executed after load balancing.

- Statistics by instance granularity, enabled via WithInstanceMW.
- Instance-Level Breaker is used to solve the single-instance exception problem. If it’s triggered, the framework will automatically retry the request.
- Note that the premise of retry is that you need to enable breaker with **WithInstanceMW**, which will be executed after load balancing.
- Threshold and **Threshold Change**

The default breaker threshold is `ErrRate: 0.5, MinSample: 200`, which means it's triggered by an error rate of 50% and requires the count of requests > 200.
The default breaker threshold is `ErrRate: 0.5, MinSample: 200`, which means its triggered by an error rate of 50% and requires the amount of requests > 200.

If you want to change the threshold, you can modify the `UpdateServiceCBConfig` and `UpdateInstanceCBConfig` in CBSuite.

## The Role of Circuit Breaker

When making RPC calls, errors are inevitable for downstream services.

When a downstream has a problem, if the upstream continues to make calls to it, it both prevents the downstream from recovering and wastes the upstream's resources.
When a downstream has a problem, if the upstream continues to make calls to it, it both prevents the downstream from recovering and wastes the upstreams resources.

To solve this problem, you can set up some dynamic switches that manually shut down calls to the downstream when it goes wrong.

Expand Down Expand Up @@ -115,16 +116,15 @@ In general, the transition of the three states is roughly as follows:
The timeout for cooling
| v
+-- detect succeed --<-[HALFOPEN]-->--+
```

### Trigger Strategies

Kitex provides three basic fuse triggering strategies by default:

- Number of consecutive errors reaches threshold (ConsecutiveTripFunc)

- Error count reaches threshold (ThresholdTripFunc)

- Error rate reaches the threshold (RateTripFunc)

Of course, you can write your own triggering strategy by implementing the TripFunc function.
Expand All @@ -145,9 +145,9 @@ During HALFOPEN, the breaker will let a request go every "interval", and after a

The process is a gradual-trial process.

Both the "interval" (DetectTimeout) and the "number" (DEFAULT_HALFOPEN_SUCCESSES) are configurable.
Both the interval (DetectTimeout) and the "number" (DEFAULT_HALFOPEN_SUCCESSES) are configurable.

## Statistics
## Statistics Algothrithm

### Default Config

Expand All @@ -171,14 +171,12 @@ As time moves, the oldest bucket in the window expires. The jitter occurs when t

As an example:

- You divide 10 seconds into 10 buckets, bucket 0 corresponds to a time of [0S, 1S), bucket 1 corresponds to [1S, 2S), ... , and bucket 9 corresponds to [9S, 10S).

- You divide 10 seconds into 10 buckets, bucket 0 corresponds to a time of [0S, 1S), bucket 1 corresponds to [1S, 2S), … , and bucket 9 corresponds to [9S, 10S).
- At 10.1S, a Succ is executed, and the following operations occur within the circuitbreaker.

- (1) detects that bucket 0 has expired and discards it;
- (2) creates a new bucket 10, corresponding to [10S, 11S);
- (3) puts that Succ into bucket 10.

- (1) detects that bucket 0 has expired and discards it;
- (2) creates a new bucket 10, corresponding to [10S, 11S);
- (3) puts that Succ into bucket 10.
- At 10.2S, you execute Successes() to query the number of successes in the window, then you get the actual statistics for [1S, 10.2S), not [0.2S, 10.2S).

Such jitter cannot be avoided if you use time-window-bucket statistics. A compromise approach is to increase the number of buckets, which can reduce the impact of jitter.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
---
title: "熔断"
date: 2023-10-23
weight: 1
keywords: ["Kitex","熔断"]
description: ""
---

# 熔断

## 介绍
Expand Down

1 comment on commit d40309a

@vercel
Copy link

@vercel vercel bot commented on d40309a Oct 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.