Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: fix over redacted log line in migration server #137915

Merged

Conversation

aa-joshi
Copy link
Contributor

@aa-joshi aa-joshi commented Dec 23, 2024

Previosuly, few log lines were getting redacted assuming that it contains
sensitive information. Support team was facing challenges during dignostics. To
address this, this patch address the log line in migration server which was
overly redacted.

Epic: CRDB-37533
Part of: CRDB-44885
Release note: None

Copy link

blathers-crl bot commented Dec 23, 2024

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@aa-joshi aa-joshi marked this pull request as ready for review December 23, 2024 10:43
@aa-joshi aa-joshi requested a review from a team as a code owner December 23, 2024 10:43
Copy link
Contributor

@arjunmahishi arjunmahishi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aa-joshi)


pkg/server/migration.go line 147 at r1 (raw file):

	}
	log.Infof(ctx, "active cluster version setting is now %s (up from %s)",
		redact.Safe(newCV.PrettyPrint()), redact.Safe(prevCV.PrettyPrint()))

I think we can directly pass newCV and prevCV without Safe or PrettyPrint. Because the underlying type implements SafeFormatter

It was getting redacted because PrettyPrint() returns string

Copy link
Contributor Author

@aa-joshi aa-joshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arjunmahishi)


pkg/server/migration.go line 147 at r1 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

I think we can directly pass newCV and prevCV without Safe or PrettyPrint. Because the underlying type implements SafeFormatter

It was getting redacted because PrettyPrint() returns string

I believe we need to wrap in PrettyPrint because it has formatting logic based on v.IsFence()
ref:

func (v Version) PrettyPrint() string {

@arjunmahishi
Copy link
Contributor

pkg/server/migration.go line 147 at r1 (raw file):

Previously, aa-joshi (Akshay Joshi) wrote…

I believe we need to wrap in PrettyPrint because it has formatting logic based on v.IsFence()
ref:

func (v Version) PrettyPrint() string {

Then I think we can change PrettyPrint and make it return redact.RedactableString
I think it's only being used for logging anyway.

@aa-joshi aa-joshi marked this pull request as draft December 23, 2024 12:32
@aa-joshi aa-joshi force-pushed the CRDB-44885_handle_log_redactions branch from 430dd79 to b8f2695 Compare January 2, 2025 10:18
@aa-joshi aa-joshi changed the title server: fix over redacted log lines server: fix over redacted log line in migration server Jan 2, 2025
Copy link
Contributor Author

@aa-joshi aa-joshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arjunmahishi)


pkg/server/migration.go line 147 at r1 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

Then I think we can change PrettyPrint and make it return redact.RedactableString
I think it's only being used for logging anyway.

PrettyPrint is invoked at multiple flows. Changing return type would increase changes significantly. As part of the task, we want to keep changes minimal so that we can backport. Let's use SafeString here.


pkg/server/auto_upgrade.go line 52 at r1 (raw file):

			clusterVersion, err := s.clusterVersion(ctx)
			if err != nil {
				log.Errorf(ctx, "unable to retrieve cluster version: %v", redact.Safe(err))

I have reverted log redaction changes for errors as we have decided to build a standard framework for it.

@aa-joshi aa-joshi marked this pull request as ready for review January 2, 2025 10:23
Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arjunmahishi)


pkg/server/migration.go line 147 at r1 (raw file):

Previously, aa-joshi (Akshay Joshi) wrote…

PrettyPrint is invoked at multiple flows. Changing return type would increase changes significantly. As part of the task, we want to keep changes minimal so that we can backport. Let's use SafeString here.

That change does not significantly change any of the non-test call sites. I wrote up a quick PR here to show it: #138353

IMO we should change PrettyPrint to return a RedactableString rather than using SafeString here.

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r2, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arjunmahishi)

Previosuly, few log lines were getting redacted assuming that it contains
sensitive information. Support team was facing challenges during dignostics. To
address this, this patch address the log line in migration server which was
overly redacted.

Epic: CRDB-37533
Part of: CRDB-44885
Release note: None
@aa-joshi aa-joshi force-pushed the CRDB-44885_handle_log_redactions branch from b8f2695 to 4570e95 Compare January 7, 2025 06:39
Copy link
Contributor Author

@aa-joshi aa-joshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arjunmahishi and @rafiss)


pkg/server/migration.go line 147 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

That change does not significantly change any of the non-test call sites. I wrote up a quick PR here to show it: #138353

IMO we should change PrettyPrint to return a RedactableString rather than using SafeString here.

Thanks for the review! updated.

Copy link
Contributor

@arjunmahishi arjunmahishi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aa-joshi and @rafiss)


pkg/clusterversion/cockroach_versions_test.go line 150 at r3 (raw file):

	}
	for _, test := range tests {
		if actual := test.cv.PrettyPrint().StripMarkers(); actual != test.exp {

We should not be getting any redaction markers if the redaction is handled correctly. If we are just using StripMarkers to resolve the type to string, we can just convert the type of exp to redact.RedactableString on line 141.

Removing StripMarkers will ensure that the test asserts the output of PrettyPrint is not redacted.

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice fix!

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arjunmahishi)


pkg/clusterversion/cockroach_versions_test.go line 150 at r3 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

We should not be getting any redaction markers if the redaction is handled correctly. If we are just using StripMarkers to resolve the type to string, we can just convert the type of exp to redact.RedactableString on line 141.

Removing StripMarkers will ensure that the test asserts the output of PrettyPrint is not redacted.

in tests, there's no strong reason to use redacted strings. IMO, it's ok to use StripMarkers here. however, it's also true that changing the type of exp should fix it and is also a simple change. i'd be fine either way,

@rafiss rafiss added backport-23.2.x Flags PRs that need to be backported to 23.2. backport-24.1.x Flags PRs that need to be backported to 24.1. backport-24.2.x Flags PRs that need to be backported to 24.2 backport-24.3.x Flags PRs that need to be backported to 24.3 labels Jan 7, 2025
@aa-joshi
Copy link
Contributor Author

aa-joshi commented Jan 8, 2025

TFTR
bors r+

@craig craig bot merged commit 257521b into cockroachdb:master Jan 8, 2025
22 checks passed
Copy link

blathers-crl bot commented Jan 8, 2025

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from 4570e95 to blathers/backport-release-23.2-137915: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 23.2.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.2.x Flags PRs that need to be backported to 23.2. backport-24.1.x Flags PRs that need to be backported to 24.1. backport-24.2.x Flags PRs that need to be backported to 24.2 backport-24.3.x Flags PRs that need to be backported to 24.3
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants