Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add log masking feature #138

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

OmarAlJarrah
Copy link
Contributor

@OmarAlJarrah OmarAlJarrah commented Dec 22, 2024

Situation

Our SDKs are responsible for logging requests and responses transmitted over the network. Some requests include fields subject to the Payment Card Industry Data Security Standard (PCI DSS). To ensure compliance and safeguard sensitive information, these fields, along with other critical data, need to be masked in logs.

Task

The task was to implement a more flexible and context-aware log masking solution in the SDK. The current implementation using the ejmask library had limitations such as static configuration and global masking behavior, which could lead to unintended masking across different client instances.

Action

To address these issues, we introduced a pattern-based masking approach that allows for more granular and configurable masking rules. This approach ensures that sensitive information is appropriately masked in logs while providing the flexibility to define masking rules specific to each client instance. Key components of the new implementation include:

  • LogMasker class: Handles pattern-based masking.
  • LogMaskingFeature class: Configures and enables log masking.
  • MaskingPatternBuilder class: Builds masking patterns.
  • Custom pattern builders for JSON field masking.

Result

The new implementation ensures that sensitive information is securely masked in logs, enhancing the overall security of the SDK. It provides the flexibility to define masking rules specific to each client instance, addressing the limitations of the previous implementation.

Test

Added the following new unit-test files:

  • code/src/test/kotlin/com/expediagroup/sdk/core/logging/masking/LogMaskerTest.kt
  • code/src/test/kotlin/com/expediagroup/sdk/core/logging/masking/LogMaskingFeatureTest.kt
  • code/src/test/kotlin/com/expediagroup/sdk/core/logging/masking/MaskingPatternBuilderTest.kt
  • code/src/test/kotlin/com/expediagroup/sdk/core/logging/masking/PatternBuildersTest.kt

Notes

NaN

@OmarAlJarrah OmarAlJarrah requested a review from a team as a code owner December 22, 2024 10:38
* @return The current instance of MaskingPatternBuilder.
*/
fun pathFields(vararg paths: List<String>) = apply {
pathFields += paths.map { it.takeLast(2) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we document this behavior in the method doc? we need to keep track of this workaround

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will this affect us? Didn't you mention you have a workaround for it?

And to make sure I understand, I wouldn't be able to mask x in a.b.c.x without masking a.x.y.z because of this limit. Right? Please add a clear example to the doc.

get() = setOf(listOf("field3", "field4"))
}

assertEquals(maskingConfiguration.globalMaskedFields.size, 2)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more assertions should be added to reflect the test name

/**
* Object implementing the PatternBasedMask interface.
*/
internal val mask = object : PatternBasedMask {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this to be an object? Couldn't we build and pass the masking configs somewhere?

import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance

@TestInstance(TestInstance.Lifecycle.PER_METHOD)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the default behaviour, I think we could/should remove the annotation.

}

@Test
fun `passed path fields are truncated to the last two elements`() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document that this test should fail once support for more elements is added.

* @return The current instance of MaskingPatternBuilder.
*/
fun pathFields(vararg paths: List<String>) = apply {
pathFields += paths.map { it.takeLast(2) }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will this affect us? Didn't you mention you have a workaround for it?

And to make sure I understand, I wouldn't be able to mask x in a.b.c.x without masking a.x.y.z because of this limit. Right? Please add a clear example to the doc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants