Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DNM] Wal debug 6.29.tikv #356

Open
wants to merge 18 commits into
base: 6.29.tikv
Choose a base branch
from

Conversation

tonyxuqqi
Copy link

To debug WAL corruption issues

Qi Xu added 6 commits February 15, 2024 18:54
Signed-off-by: Qi Xu <[email protected]>
Signed-off-by: Qi Xu <[email protected]>
Signed-off-by: Qi Xu <[email protected]>
Signed-off-by: Qi Xu <[email protected]>
fix
Signed-off-by: Qi Xu <[email protected]>
Qi Xu added 2 commits March 13, 2024 14:16
Signed-off-by: Qi Xu <[email protected]>
@mittalrishabh
Copy link

@tonyxuqqi What is the WAL recovery mode in tikv ? I guess that it should be kTolerateCorruptedTailRecords.

@mittalrishabh
Copy link

which format is tikv using. Seems like rocksdb wal has two formats : legacy and recycle. Recycle format has more debug information (log number) . https://github.com/facebook/rocksdb/wiki/Write-Ahead-Log-File-Format

Qi Xu added 2 commits March 14, 2024 14:23
Signed-off-by: Qi Xu <[email protected]>
Signed-off-by: Qi Xu <[email protected]>
"Start writing to WAL: [%" PRIu64 " ]",
log_writer->get_log_number());
}
if (log_writer->get_log_number() != logs_.back().number) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tonyxuqqi : At the beginning of the write, writer will reference the last WAL only. The race condition happens, while the write is ongoing. We should check this condition after the write.

@@ -1488,7 +1501,7 @@ IOStatus DBImpl::WriteToWAL(const WriteThread::WriteGroup& write_group,

uint64_t log_size;
io_s = WriteToWAL(*merged_batch, log_writer, log_used, &log_size,
log_file_number_size);
log_file_number_size, 1);
if (to_be_cached_state) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also add a lot when the fsync happens in writeToWAL

io_s = log.writer->file()->Sync(immutable_db_options_.use_fsync);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants