Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spanner MR CMEK Integration #11319

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b43da49
Update Database.yaml
panerorenn9541 Jul 31, 2024
a7e959f
Change kms_key_names to kmsKeyNames in description
panerorenn9541 Jul 31, 2024
0f58e19
Update resource_spanner_database_test.go.erb
panerorenn9541 Aug 1, 2024
be9a029
Merge pull request #1 from panerorenn9541/mr-cmek-integration
panerorenn9541 Aug 1, 2024
4a0d02f
Merge branch 'GoogleCloudPlatform:main' into main
panerorenn9541 Aug 1, 2024
50929aa
Update Database.yaml
panerorenn9541 Aug 5, 2024
aed43a4
Merge branch 'GoogleCloudPlatform:main' into main
panerorenn9541 Aug 5, 2024
0a04792
Add item_type to kmsKeyNames array.
panerorenn9541 Aug 7, 2024
7865541
Merge branch 'GoogleCloudPlatform:main' into main
panerorenn9541 Aug 7, 2024
8687ccc
Remove quotations from key names in resource_spanner_database_test.go…
panerorenn9541 Aug 9, 2024
8b8764a
Merge branch 'GoogleCloudPlatform:main' into main
panerorenn9541 Aug 9, 2024
f58f858
merge-with-main
ScottSuarez Oct 8, 2024
3b28000
migrate pull request and fix test case/documentation
ScottSuarez Oct 8, 2024
95d98d7
Don't read kms_key_names if kms_key_name is set
ScottSuarez Oct 8, 2024
37586d6
Update permadiff.md
panerorenn9541 Oct 10, 2024
5795610
Update permadiff.md
panerorenn9541 Oct 11, 2024
084a6a0
Update Database.yaml
panerorenn9541 Oct 11, 2024
0c0c9e5
Update mmv1/templates/terraform/custom_flatten/spanner_database_kms_k…
panerorenn9541 Oct 11, 2024
b416e1d
Update spanner_database_kms_key_names.go.tmpl
panerorenn9541 Oct 11, 2024
e2144b4
Update resource_spanner_database_test.go.tmpl
panerorenn9541 Oct 11, 2024
2887416
Update resource_spanner_database_test.go.tmpl
panerorenn9541 Oct 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 34 additions & 6 deletions docs/content/develop/permadiff.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,15 +234,29 @@ Add a [custom flattener]({{< ref "/develop/custom-code#custom_flatten" >}}) for

```go
func flatten{{$.GetPrefix}}{{$.TitlelizeProperty}}(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
configValue := d.Get("path.0.to.0.parent_field.0.nested_field").([]string)
rawConfigValue := d.Get("path.0.to.0.parent_field.0.nested_field")

sorted, err := tpgresource.SortStringsByConfigOrder(configValue, v.([]string))
// Convert config value to []string
configValue, err := tpgresource.InterfaceSliceToStringSlice(rawConfigValue)
if err != nil {
log.Printf("[ERROR] Failed to convert config value: %s", err)
return v
}

// Convert v to []string
apiStringValue, err := tpgresource.InterfaceSliceToStringSlice(v)
if err != nil {
log.Printf("[ERROR] Failed to convert API value: %s", err)
return v
}

sortedStrings, err := tpgresource.SortStringsByConfigOrder(configValue, apiStringValue)
if err != nil {
log.Printf("[ERROR] Could not sort API response value: %s", err)
return v
}

return sorted.(interface{})
return sortedStrings
}
```
{{< /tab >}}
Expand All @@ -251,15 +265,29 @@ Define resource-specific functions in your service package, for example at the t

```go
func flattenResourceNameFieldName(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
configValue := d.Get("path.0.to.0.parent_field.0.nested_field").([]string)
rawConfigValue := d.Get("path.0.to.0.parent_field.0.nested_field")

// Convert config value to []string
configValue, err := tpgresource.InterfaceSliceToStringSlice(rawConfigValue)
if err != nil {
log.Printf("[ERROR] Failed to convert config value: %s", err)
return v
}

// Convert v to []string
apiStringValue, err := tpgresource.InterfaceSliceToStringSlice(v)
if err != nil {
log.Printf("[ERROR] Failed to convert API value: %s", err)
return v
}

sorted, err := tpgresource.SortStringsByConfigOrder(configValue, v.([]string))
sortedStrings, err := tpgresource.SortStringsByConfigOrder(configValue, apiStringValue)
if err != nil {
log.Printf("[ERROR] Could not sort API response value: %s", err)
return v
}

return sorted.(interface{})
return sortedStrings
}
```
{{< /tab >}}
Expand Down
16 changes: 15 additions & 1 deletion mmv1/products/spanner/Database.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,22 @@ properties:
description: |
Fully qualified name of the KMS key to use to encrypt this database. This key must exist
in the same location as the Spanner Database.
required: true
immutable: true
exactly_one_of:
- encryption_config.0.kms_key_name
- encryption_config.0.kms_key_names
- name: 'kmsKeyNames'
type: Array
description: |
Fully qualified name of the KMS keys to use to encrypt this database. The keys must exist
in the same locations as the Spanner Database.
immutable: true
custom_flatten: templates/terraform/custom_flatten/spanner_database_kms_key_names.go.tmpl
item_type:
type: String
exactly_one_of:
- encryption_config.0.kms_key_name
- encryption_config.0.kms_key_names
- name: 'databaseDialect'
type: Enum
description: |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
func flatten{{$.GetPrefix}}{{$.TitlelizeProperty}}(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
// Ignore `kms_key_names` if `kms_key_name` is set, because that field takes precedence.
_, kmsNameSet := d.GetOk("encryption_config.0.kms_key_name")
if kmsNameSet {
return nil
}

rawConfigValue := d.Get("encryption_config.0.kms_key_names")

// Convert config value to []string
configValue, err := tpgresource.InterfaceSliceToStringSlice(rawConfigValue)
if err != nil {
log.Printf("[ERROR] Failed to convert config value: %s", err)
return v
}

// Convert v to []string
apiStringValue, err := tpgresource.InterfaceSliceToStringSlice(v)
if err != nil {
log.Printf("[ERROR] Failed to convert API value: %s", err)
return v
}

sortedStrings, err := tpgresource.SortStringsByConfigOrder(configValue, apiStringValue)
if err != nil {
log.Printf("[ERROR] Could not sort API response value: %s", err)
return v
}

return sortedStrings
}
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ func TestAccSpannerDatabase_cmek(t *testing.T) {
ResourceName: "google_spanner_database.database",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"ddl", "deletion_protection"},
ImportStateVerifyIgnore: []string{"ddl", "deletion_protection", "encryption_config.0.kms_key_names"},
},
},
})
Expand Down Expand Up @@ -605,4 +605,110 @@ resource "google_project_service_identity" "ck_sa" {

`, context)
}

func TestAccSpannerDatabase_mrcmek(t *testing.T) {
acctest.SkipIfVcr(t)
t.Parallel()

kms1 := acctest.BootstrapKMSKeyWithPurposeInLocationAndName(t, "ENCRYPT_DECRYPT", "us-central1", "tf-mr-cmek-test-key-us-central1")
kms2 := acctest.BootstrapKMSKeyWithPurposeInLocationAndName(t, "ENCRYPT_DECRYPT", "us-east1", "tf-mr-cmek-test-key-us-east1")
kms3 := acctest.BootstrapKMSKeyWithPurposeInLocationAndName(t, "ENCRYPT_DECRYPT", "us-east4", "tf-mr-cmek-test-key-us-east4")
context := map[string]interface{}{
"random_suffix": acctest.RandString(t, 10),
"key_ring1": kms1.KeyRing.Name,
"key_name1": kms1.CryptoKey.Name,
"key_ring2": kms2.KeyRing.Name,
"key_name2": kms2.CryptoKey.Name,
"key_ring3": kms3.KeyRing.Name,
"key_name3": kms3.CryptoKey.Name,
}

acctest.VcrTest(t, resource.TestCase{
PreCheck: func() { acctest.AccTestPreCheck(t) },
ProtoV5ProviderFactories: acctest.ProtoV5ProviderBetaFactories(t),
CheckDestroy: testAccCheckSpannerDatabaseDestroyProducer(t),
Steps: []resource.TestStep{
{
Config: testAccSpannerDatabase_mrcmek(context),
},
{
ResourceName: "google_spanner_database.database",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"ddl", "deletion_protection"},
},
},
})
}

func testAccSpannerDatabase_mrcmek(context map[string]interface{}) string {
return acctest.Nprintf(`
resource "google_spanner_instance" "main" {
provider = google-beta
config = "nam3"
display_name = "main-instance1"
num_nodes = 1
}

resource "google_spanner_database" "database" {
provider = google-beta
instance = google_spanner_instance.main.name
name = "tf-test-mrcmek-db%{random_suffix}"
ddl = [
"CREATE TABLE t1 (t1 INT64 NOT NULL,) PRIMARY KEY(t1)",
"CREATE TABLE t2 (t2 INT64 NOT NULL,) PRIMARY KEY(t2)",
]

encryption_config {
kms_key_names = [
"%{key_name1}",
"%{key_name2}",
"%{key_name3}",
]
}

deletion_protection = false

depends_on = [google_kms_crypto_key_iam_binding.crypto_key1,
google_kms_crypto_key_iam_binding.crypto_key2,
google_kms_crypto_key_iam_binding.crypto_key3,]
}

resource "google_kms_crypto_key_iam_binding" "crypto_key1" {
crypto_key_id = "%{key_name1}"
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
members = [
google_project_service_identity.ck_sa.member,
]
}

resource "google_kms_crypto_key_iam_binding" "crypto_key2" {
crypto_key_id = "%{key_name2}"
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
members = [
google_project_service_identity.ck_sa.member,
]
}

resource "google_kms_crypto_key_iam_binding" "crypto_key3" {
crypto_key_id = "%{key_name3}"
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
members = [
google_project_service_identity.ck_sa.member,
]
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove these IAM roles? I'll manually add them once done. Then we should be good to go here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the depends_on = need to go too?


data "google_project" "project" {
provider = google-beta
}

resource "google_project_service_identity" "ck_sa" {
provider = google-beta
project = data.google_project.project.project_id
service = "spanner.googleapis.com"
}

`, context)
}

{{- end }}
19 changes: 19 additions & 0 deletions mmv1/third_party/terraform/tpgresource/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,25 @@ func ExpandStringMap(d TerraformResourceData, key string) map[string]string {
return ConvertStringMap(v.(map[string]interface{}))
}

// InterfaceSliceToStringSlice converts a []interface{} containing strings to []string
func InterfaceSliceToStringSlice(v interface{}) ([]string, error) {
ScottSuarez marked this conversation as resolved.
Show resolved Hide resolved
interfaceSlice, ok := v.([]interface{})
if !ok {
return nil, fmt.Errorf("expected []interface{}, got %T", v)
}

stringSlice := make([]string, len(interfaceSlice))
for i, item := range interfaceSlice {
strItem, ok := item.(string)
if !ok {
return nil, fmt.Errorf("expected string, got %T at index %d", item, i)
}
stringSlice[i] = strItem
}

return stringSlice, nil
}

// SortStringsByConfigOrder takes a slice of map[string]interface{} from a TF config
// and API data, and returns a new slice containing the API data, reorderd to match
// the TF config as closely as possible (with new items at the end of the list.)
Expand Down
Loading