Skip to content
This repository has been archived by the owner on Feb 6, 2024. It is now read-only.

refactor: make create table idempotent #278

Closed
wants to merge 6 commits into from

Conversation

ZuLiangWang
Copy link
Contributor

@ZuLiangWang ZuLiangWang commented Nov 16, 2023

Rationale

Store the intermediate data for table creation through procedures to support idempotent table creation.

Detailed Changes

  • Store the intermediate data for table creation through procedure.
  • Supports retrying from the upper layer to complete failed table creation procedures.

Test Plan

Pass all unit tests and integration tests.

@ZuLiangWang ZuLiangWang changed the title make create table idempotent refactor: make create table idempotent Nov 16, 2023
@ZuLiangWang ZuLiangWang force-pushed the refactor_create_table branch 3 times, most recently from faa73fd to 90069e5 Compare November 20, 2023 03:26
@ZuLiangWang ZuLiangWang marked this pull request as ready for review November 20, 2023 03:29
@ZuLiangWang ZuLiangWang force-pushed the refactor_create_table branch 2 times, most recently from f1d0cdc to edc14a5 Compare November 21, 2023 03:11
## Rationale
Due to misconfiguration of the ProcedureID Allocator storage path in
ETCD, multiple clusters will share the same ProcedureID Allocator, which
will cause ID to be wasted.

## Detailed Changes
* Fix wrong ID allocator storage path.

## Test Plan
Pass CI.
## Rationale
The current design of the procedure storage module is unreasonable. The
data will be stored permanently and will not be cleared. This will cause
the storage space of `ETCD` to be filled up quickly. We need to redesign
the interface of procedure storage.

## Detailed Changes
* Add `ttl` config for `CreateOrUpdate`.
* Add `Delete` interface, it used to delete history procedure data and
current procedure data.

## Test Plan
Pass all unit tests and integration tests.
@ZuLiangWang ZuLiangWang force-pushed the refactor_create_table branch from edc14a5 to baa64ee Compare December 5, 2023 11:47
@ZuLiangWang ZuLiangWang closed this Dec 5, 2023
@ZuLiangWang
Copy link
Contributor Author

We are currently developing based on the dev branch. Close this PR and re-submit to the dev branch.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant