Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DNM,Cloud] Cloud master #240

Open
wants to merge 136 commits into
base: lrz/test-111
Choose a base branch
from
Open

[DNM,Cloud] Cloud master #240

wants to merge 136 commits into from

Conversation

CalvinNeo
Copy link
Member

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

lidezhu and others added 30 commits November 15, 2022 17:47
@lidezhu
alpha version
1b55bc4
@CalvinNeo
introduce cache mem info
1e978b9 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
partially run
758e22f 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
it can work
2efbf19 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix test bug
c3afc97 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
use snapshot to compose raft apply state
e6228f5 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
do not copy all meta
f529bce 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fmt
9da03d1 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
Merge branch 'raftstore-proxy' into another-fast-add-demo
7a93e5f
@CalvinNeo
fast add peer
a652cbd 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix error router
22afec3 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
Merge remote-tracking branch 'upstream/raftstore-proxy' into another-…
2da4c93 
…fast-add-demo
@CalvinNeo
fix merge
0a5c25b 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix logs
1c11747 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
f
2fb1939 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
f
fb0917b 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
f
7da9b92 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
f
fa6099c 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
Merge fast add peer (#230)
1b335a3
@CalvinNeo
add test for paused
660eb73 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix ci
39f6a14 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix ci
fffc345 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix tests
59e1616 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix tests
eb2c930 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
fix tests
a9ff447 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
support re add peer
4929749 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
re fmt
e4d6453 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
Add support for recover and re-add peer (#231)
391ba67
@CalvinNeo
add tests for apply snapshot
d6617f9 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
add split and merge test
8b7abda 
Signed-off-by: CalvinNeo <[email protected]>
Connor1996 and others added 19 commits January 20, 2023 22:43
@Connor1996
batch-system: add priority scheduling for batch system (tikv#14065)
9726e56 
ref tikv#13730

Support priority-based scheduling for the apply batch system.

Signed-off-by: Connor1996 <[email protected]>
@CalvinNeo
[Cloud] Merge raftstore-proxy master(PR #249) (#259)
fcaa14a
@CalvinNeo
Merge remote-tracking branch 'tikv/master' into merge-tikv-release-6.6-2
5e32274
@andreid-db
config: allow starting TiKV nodes with <1 CPU (tikv#14084)
c353910 
close tikv#13586, close tikv#13752, ref tikv#14017

Signed-off-by: Andrei Dragus <[email protected]>
@HuSharp @ti-chi-bot
pd_client: replace PD_REQUEST_HISTOGRAM_VEC with static metrics (tikv…
68710b9 
…#14087)

close tikv#14086

PD_REQUEST_HISTOGRAM_VEC can be changed to use static metrics to improve performance.

Signed-off-by: husharp <[email protected]>

Co-authored-by: Ti Chi Robot <[email protected]>
@CalvinNeo
adapt tikv's feature
1f04c72 
Signed-off-by: CalvinNeo <[email protected]>
@LittleFall
copr: (refactor) Lift heap struct out from top_n_executor (tikv#14096)
b1936e6 
ref tikv#13936

Signed-off-by: Zhi Qi <[email protected]>
@JmPotato @ti-chi-bot
raftstore: allow the read request with a smaller ts during flashback (t…
0ce3485 
…ikv#14088)

close tikv#14045

- Store the flashback `start_ts` in region meta.
- Allow the read request with a smaller ts during flashback.

Signed-off-by: JmPotato <[email protected]>

Co-authored-by: Ti Chi Robot <[email protected]>
@iosmanthus
import: sst_importer support download SST and rewrite into keyspace d…
7ec73fd 
…ata. (tikv#14046)

ref tikv#12999

import: sst_importer support download SST and rewrite into keyspace data.

Signed-off-by: iosmanthus <[email protected]>
@CalvinNeo
Merge remote-tracking branch 'tikv/master' into merge-tikv-release-6.6-2
fc18cfa
@tonyxuqqi
rocksdb: reduce rocksdb block size to 16KB (tikv#14053)
ec2f4dc 
close tikv#14052

The writecf and defaultcf's default block size is changed to 16KB to improve read performance (reduce read amplification)

Signed-off-by: qi.xu <[email protected]>

Co-authored-by: qi.xu <[email protected]>
@tonyxuqqi @ti-chi-bot
storage: add an alias partitioned-raft-kv for RaftKv2 (tikv#14083)
15d6040 
ref tikv#12842

add an alias partitioned-raft-kv for RaftKv2

Signed-off-by: qi.xu <[email protected]>

Co-authored-by: qi.xu <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
@CalvinNeo
Merge pull request #262 from CalvinNeo/merge-tikv-release-6.6-2
6444536 
Merge master about 6.6(7ec73fd)
@overvenus @ti-chi-bot
resolved_ts: reduce network traffic by filter regions (tikv#14098)
23a2288 
close tikv#14092

resolved_ts: reduce network traffic by filter regions

Signed-off-by: Neil Shen <[email protected]>

Co-authored-by: Ti Chi Robot <[email protected]>
@BusyJay @ti-chi-bot
raftstore-v2: fix peer not cleanup when it replicates more logs (tikv…
a33eb2d 
…#14101)

ref tikv#12842

If it accepts more logs than conf remove itself, applied_index == commit_index
will never be true. So we should check if it's a tombstone already first.

Signed-off-by: Jay Lee <[email protected]>

Co-authored-by: Ti Chi Robot <[email protected]>
@you06
cop: handle unset scan details in store batch (tikv#14102)
9c0df6d 
close tikv#14109

Signed-off-by: you06 <[email protected]>
@nolouch
resource_control: unify wru/rru to ru (tikv#14121)
db14c53 
close tikv#14120

resource_control: unify wru/rru to ru

Signed-off-by: nolouch <[email protected]>
@CalvinNeo
[Cloud] Add tests for assertion of single replica migration (#264)
df5cc1d 
Signed-off-by: CalvinNeo <[email protected]>
@HuSharp
pd_client: fix item value type (tikv#14106)
d1d2920 
close tikv#14104

We need to use the new field to support item value as bytes to avoid proto string check failures.

Signed-off-by: husharp <[email protected]>
@CalvinNeo CalvinNeo changed the base branch from raftstore-proxy to lrz/test-111 February 2, 2023 08:36
BusyJay and others added 9 commits February 3, 2023 13:57
@BusyJay
raftstore-v2: add tablet logger and update dep (tikv#14129)
3791560 
ref tikv#12842

- Update raft-engine to fix data corruption during restart
- Add tablet logger so we can know which tablet the logs belongs to

Signed-off-by: Jay Lee <[email protected]>
@CalvinNeo
raftstore: Observe when receive raft message (tikv#14043)
c8c1ca8 
ref tikv#13855

Introduce observers when receive raft message.

Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
Merge remote-tracking branch 'tikv/master' into merge-tikv-6.6-3
17a13b0
@CalvinNeo
adapt to tikv master
31a0fed 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
[Cloud] Fix before merge master (#265)
b96048f
@CalvinNeo
Merge pull request #266 from CalvinNeo/merge-tikv-6.6-3
46ec025 
Merge TiKV master(c8c1ca8)
@CalvinNeo
Merge remote-tracking branch 'upstream/raftstore-proxy' into merge-mv…
964dbb4 
…p-to-tikv-master
@CalvinNeo
merge tikv upstream/raftstore-proxy
4f4fcbb 
Signed-off-by: CalvinNeo <[email protected]>
@CalvinNeo
Merge pull request #267 from CalvinNeo/merge-mvp-to-tikv-master
c1e23d6 
[Cloud] Merge raftstore-proxy with new TiKV observer into cloud branch
@CLAassistant
Copy link

CLAassistant commented Feb 3, 2023 •
edited
Loading

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
4 out of 16 committers have signed the CLA.

✅ HuSharp
✅ YangKeao
✅ CalvinNeo
✅ LittleFall
❌ tabokie
❌ YuJuncen
❌ andreid-db
❌ ethercflow
❌ Connor1996
❌ iosmanthus
❌ overvenus
❌ BusyJay
❌ JmPotato
❌ tonyxuqqi
❌ nolouch
❌ you06
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.