Skip to content

Commit

Permalink
docs: add data masking 3 (#481)
Browse files Browse the repository at this point in the history
* docs: optimize part 1&2

* update

* docs: add masking-3
  • Loading branch information
OP3ratordec authored Dec 5, 2024
1 parent fd2d4de commit a5b8e7a
Show file tree
Hide file tree
Showing 3 changed files with 101 additions and 10 deletions.
8 changes: 4 additions & 4 deletions content/docs/tutorials/github-action-data-masking-part1.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ By using GitHub Actions with Bytebase API, you can implement policy-as-code to a
This is Part 1 of our tutorial series on implementing automated database masking using GitHub Actions:

- Part 1: Column masking and masking exception with GitHub Actions (this one)
- Part 2: Masking Algorithm with GitHub Actions
- Part 3: Data Classification and Global Masking with GitHub Actions
- Part 2: [Masking Algorithm with GitHub Actions](/docs/tutorials/github-action-data-masking-part2)
- Part 3: [Data Classification and Global Masking with GitHub Actions](/docs/tutorials/github-action-data-masking-part3)

## Overview

Expand Down Expand Up @@ -102,7 +102,7 @@ In the GitHub workflow, find the step `Apply column masking`, which will apply t
--data @"$CHANGED_FILE")
```

By changing the files `masking/databases/**/**/column-masking.json`, create a PR and then merge, the change will be applied to the database.
By changing file `masking/databases/**/**/column-masking.json`, create a PR and then merge, the change will be applied to the database.

Log in Bytebase console, at the workspace level, click **Data Access > Data Masking**. Click **Explicit Masked Columns**, you can see the column masking is applied to the database.

Expand All @@ -121,7 +121,7 @@ In the GitHub workflow, find the step `Apply masking exception`, which will appl
--data @"$CHANGED_FILE")
```

By changing the files `masking/projects/**/masking-exception.json`, create a PR and then merge, the change will be applied to the database.
By changing file `masking/projects/**/masking-exception.json`, create a PR and then merge, the change will be applied to the database.

Log in Bytebase console, go to the project `Sample Project`, click **Database > Masking Access**, you can see the masking exception is applied to the database.

Expand Down
24 changes: 18 additions & 6 deletions content/docs/tutorials/github-action-data-masking-part2.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ In the [previous tutorial](/docs/tutorials/github-action-data-masking-part1), yo
This is Part 2 of our tutorial series on implementing automated database masking using GitHub Actions:

- Part 1: [Column masking and masking exception with GitHub Actions](/docs/tutorials/github-action-data-masking-part1)
- Part 2: Masking Algorithm with GitHub Actions (this one)
- Part 3: Data Classification and Global Masking with GitHub Actions
- Part 2: Masking Algorithm with GitHub Actions(this one)
- Part 3: [Data Classification and Global Masking with GitHub Actions](/docs/tutorials/github-action-data-masking-part3)

## Overview

Expand All @@ -36,16 +36,20 @@ The complete code for this tutorial is available at: [database-security-github-a

</HintBlock>

This tutorial skips the setup part, if you haven't set up the Bytebase and GitHub Action, please refer to the [previous tutorial](/docs/tutorials/github-action-data-masking-part1).
This tutorial skips the setup part, if you haven't set up the Bytebase and GitHub Action, please follow **Setup Instructions** section in the [previous tutorial](/docs/tutorials/github-action-data-masking-part1).

## Masking Algorithm

You may customize your own [data masking algorithm](/docs/security/data-masking/masking-algorithm/) with the help of a predefined masking type, such as Full mask, Range mask, MD5 mask and Inner/Outer mask.

In Bytebase console, go to **Data Access > Data Masking**, click **Masking Algorithm** and click **Add**. You can create a new masking algorithm with a name and description, and later it can be used in the definition of semantic types.
### In Bytebase console

Go to **Data Access > Data Masking**, click **Masking Algorithm** and click **Add**. You can create a new masking algorithm with a name and description, and later it can be used in the definition of semantic types.

![bb-masking-algorithm](/content/docs/tutorials/github-action-data-masking-part2/bb-masking-algorithm.webp)

### In GitHub Workflow

In the GitHub workflow `bb-masking-2.yml`, find the step `Apply masking algorithm`, which will apply the masking algorithm to the database via API. All the masking algorithms should be defined in one file in the root directory of `masking/masking-algorithm.json`. The code it calls Bytebase API is as follows:

```bash
Expand All @@ -55,15 +59,21 @@ response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/sett
--data @"$CHANGED_FILE")
```

By changing file `masking/masking-algorithm.json`, you can apply the masking algorithm to the database. Go to Bytebase console, click **Data Access > Data Masking**, go to **Masking Algorithm** page, you can see the masking algorithm is applied to the database.

## Semantic Type

You may define [semantic types](/docs/security/data-masking/semantic-types/) and apply them to columns of different tables. Columns with the same semantic type will be masked with the same masking algorithm. For example, you may define a semantic type `mobile` and apply it to all the columns of phone number. Then you can define a masking algorithm `range 4-10` for the partial level masking for semantic type `mobile`.

In Bytebase console, go to **Data Access > Data Masking**, click **Semantic Types** and click **Add**. You can create a new semantic type with a name and description, and select the masking algorithm.
### In Bytebase Console

Go to **Data Access > Data Masking**, click **Semantic Types** and click **Add**. You can create a new semantic type with a name and description, and select the masking algorithm.

![bb-semantic-types](/content/docs/tutorials/github-action-data-masking-part2/bb-semantic-types.webp)

In the GitHub workflow, find the step `Apply semantic type`, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as `masking/semantic-type.json`. The code it calls Bytebase API is as follows:
### In GitHub Workflow

Find the step `Apply semantic type`, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as `masking/semantic-type.json`. The code it calls Bytebase API is as follows:

```bash
response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/settings/bb.workspace.semantic-types?allow_missing=true" \
Expand All @@ -72,6 +82,8 @@ response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/sett
--data @"$CHANGED_FILE")
```

By changing file `masking/semantic-type.json`, you can apply the semantic type to the database. Go to Bytebase console, click **Data Access > Data Masking**, go to **Semantic Types** page, you can see the semantic type is applied to the database.

## Next Steps

Now you have successfully applied data masking algorithm and semantic type using GitHub Actions and Bytebase API. In the next part of this tutorial, you'll learn how to use data classification and global masking with GitHub Actions. Stay tuned!
79 changes: 79 additions & 0 deletions content/docs/tutorials/github-action-data-masking-part3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
title: 'Applying Data Masking with GitHub Actions - Part 3'
author: Dec
updated_at: 2024/12/04 18:00
tags: Tutorial
integrations: General, API
level: Advanced
estimated_time: '30 mins'
description: 'Learn how to use data classification and global masking policy using GitHub Actions and Bytebase API'
---

<IncludeBlock url="/docs/share/tutorials/api-preface"></IncludeBlock>

In the [previous tutorial](/docs/tutorials/github-action-data-masking-part2), you learned how to customize both the masking algorithm and semantic types. In this tutorial, we will explore how to use data classification and global masking policy.

---

This is Part 3 of our tutorial series on implementing automated database masking using GitHub Actions:

- Part 1: [Column masking and masking exception with GitHub Actions](/docs/tutorials/github-action-data-masking-part1)
- Part 2: [Masking Algorithm with GitHub Actions](/docs/tutorials/github-action-data-masking-part2)
- Part 3: Data Classification and Global Masking with GitHub Actions(this one)

## Overview

In this tutorial, you'll learn how to automate data classification and global masking policy using GitHub Actions and the Bytebase API. This integration allows you to:

- Manage data classification and global masking policy as code
- Automatically apply masking policies when PRs are merged

<HintBlock type="info">

The complete code for this tutorial is available at: [database-security-github-actions-example](https://github.com/bytebase/database-security-github-actions-example)

</HintBlock>

This tutorial skips the setup part, if you haven't set up the Bytebase and GitHub Action, please follow **Setup Instructions** section in the [previous tutorial](/docs/tutorials/github-action-data-masking-part1).

## Data Classification

### In Bytebase Console

Follow [Data Classification](/docs/security/data-classification/).

### In GitHub Workflow

Find the step `Apply classification`, which will apply the classification to the database via API. All the classifications should be defined in one file in the root directory as `masking/classification.json`. The code it calls Bytebase API is as follows:

```bash
response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/settings/bb.workspace.classification" \
--header "Authorization: Bearer ${BYTEBASE_TOKEN}" \
--header "Content-Type: application/json" \
--data @"$CHANGED_FILE")
```

By changing file `masking/classification.json`, creating a PR and merging, you can apply the classification to the database. Go to Bytebase console, click **Data Access > Data Classification**, go to **Classification** page, you can see the classification is applied to the database.

## Global Masking Rule

### In Bytebase Console

Follow [Global Masking Rule](/docs/security/data-masking/global-masking-rule/).

### In GitHub Workflow

Find the step `Apply global masking rule`, which will apply the global masking rule to the database via API. All the global masking rules should be defined in one file in the root directory as `masking/global-masking-rule.json`. The code it calls Bytebase API is as follows:

```bash
response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/settings/bb.workspace.global_masking_rule?allow_missing=true&update_mask=payload" \
--header "Authorization: Bearer ${BYTEBASE_TOKEN}" \
--header "Content-Type: application/json" \
--data @"$CHANGED_FILE")
```

By changing file `masking/global-masking-rule.json`, creating a PR and merge, you can apply the global masking rule to the database. Go to Bytebase console, click **Data Access > Data Masking**, go to **Global Masking Rule** page, you can see the global masking rule is applied to the database.

## Summary

Through out this tutorial series, you have learned how to automate database masking policies, customize both the masking algorithm and semantic types, and use data classification and global masking policy using GitHub Actions and Bytebase API.

0 comments on commit a5b8e7a

Please sign in to comment.