Enforce mandatory tags for all tagging supported resources

[gsund89]

My requirements are:

• Enforce few mandatory tags (say ApplicationName, ApplicationOwner, SupportContact, Environment & CostCenter) to all tagging supported resources. cfn-guard should raise error if any of these tags are found missing.
• Developers can add any number of tags other than the mandatory tags. cfn-guard should not fail if such tags are defined.

Below shown is a sample ruleset that I used to validate the mandatory tags for VPC.

AWS::EC2::VPC Tags == /.*"Key":"Environment".*/ << the mandatory Tag: Environment is not specified
AWS::EC2::VPC Tags == /.*"Key":"ApplicationName".*/ << the mandatory Tag: ApplicationName is not specified
AWS::EC2::VPC Tags == /.*"Key":"SupportContact".*/ << the mandatory Tag: SupportContact is not specified
AWS::EC2::VPC Tags == /.*"Key":"ApplicationOwner".*/ << the mandatory Tag: ApplicationOwner is not specified
AWS::EC2::VPC Tags == /.*"Key":"CostCenter".*/ << the mandatory Tag: CostCenter is not specified

By evaluating a CFT which contains only tag keys Service, Name and ApplicationName, using this above ruleset, I received the below output.

[VPC] failed because [Tags] is [[{"Key":"Service","Value":"VPC"},{"Key":"Name","Value":"DemoVPC"},{"Key":"ApplicationName","Value":"webapp"}]] and the mandatory Tag: ApplicationOwner is not specified
[VPC] failed because [Tags] is [[{"Key":"Service","Value":"VPC"},{"Key":"Name","Value":"DemoVPC"},{"Key":"ApplicationName","Value":"webapp"}]] and the mandatory Tag: CostCenter is not specified
[VPC] failed because [Tags] is [[{"Key":"Service","Value":"VPC"},{"Key":"Name","Value":"DemoVPC"},{"Key":"ApplicationName","Value":"webapp"}]] and the mandatory Tag: Environment is not specified
[VPC] failed because [Tags] is [[{"Key":"Service","Value":"VPC"},{"Key":"Name","Value":"DemoVPC"},{"Key":"ApplicationName","Value":"webapp"}]] and the mandatory Tag: SupportContact is not specified

Got two questions here:

1. I have used regex in ruleset to validate tags. Is there any other method suggested for checking the existence of mandatory tags?
2. As per my understanding, cfn-guard currently supports only adding rulesets for each resource type individually. Is there a way to apply this rule set globally for all resources?

[PatMyron]

Keep in mind tagging is actually one of the more difficult enforcement scenarios to statically analyze since these tags are usually passed in at the stack level rather than being present in the template itself (create-stack / update-stack --tags)

That could also be an easier way to enforce those tags on all those resource types if you can make sure they're just passed in at the stack-level, especially since most resource types don't currently support Tags properties and Guard doesn't inherently know which ones yet:

CloudformationSchemas $ grep -L '"Tags"' * | wc -l
375 # resource types that currently don't support Tags

---

I like the general idea of regular expressions for types in rules though

[gsund89]

Keep in mind tagging is actually one of the more difficult enforcement scenarios to statically analyze since these tags are usually passed in at the stack level rather than being present in the template itself (create-stack / update-stack --tags)

That could also be an easier way to enforce those tags on all those resource types if you can make sure they're just passed in at the stack-level

I like the general idea of regular expressions for types in rules though

Yes, that has to be taken care by whoever is launching the stack, though. And to ensure that, we might need some other mechanism.

[vennemp]

I'm running into this issue as well - certain resources will require different tags - for instance for AWS Backup. So defining the tags at a stack level doesn't make the best sense. We are using AWS Config to capture non-compliant resources but since everything is going done thru CI/CD and IaC it would be nice to enforce these policies at creation.

Other than that, really loving this product! Great job.

[ghost]

@nathanataws - I was wondering in which language the rulesets are written?
is it Rust? 🤔

[PatMyron]

@nathanataws - I was wondering in which language the rulesets are written?
is it Rust? 🤔

ruleset syntax is described here

[nathanataws]

The problem with the rule set above is that the lines are effectively contradicting each other. They're saying that every tag must be all of those things (which is impossible).

The best way to check all of those tags would most likely be to either |OR| the existing rules together or find a way to express the position of the value you want to check without a regex and then have all the acceptable values in a list.

Eg

AWS::EC2::VPC Tags.*.Key in [x, y, z]

The downside to that approach is that you won't be able to use those different custom messages. For that, you'll have to use the |OR| as mentioned above.

[gsund89]

The problem with the rule set above is that the lines are effectively contradicting each other. They're saying that every tag must be all of those things (which is impossible).

The best way to check all of those tags would most likely be to either |OR| the existing rules together or find a way to express the position of the value you want to check without a regex and then have all the acceptable values in a list.

Eg

AWS::EC2::VPC Tags.*.Key in [x, y, z]

The downside to that approach is that you won't be able to use those different custom messages. For that, you'll have to use the |OR| as mentioned above.

What do you mean by "effectively contradicting each other". Could you please elaborate. Because the above rule set was working fine as per our requirement.

[johnttompkins]

The problem with the rule set above is that the lines are effectively contradicting each other. They're saying that every tag must be all of those things (which is impossible).
The best way to check all of those tags would most likely be to either |OR| the existing rules together or find a way to express the position of the value you want to check without a regex and then have all the acceptable values in a list.
Eg
AWS::EC2::VPC Tags.*.Key in [x, y, z]
The downside to that approach is that you won't be able to use those different custom messages. For that, you'll have to use the |OR| as mentioned above.

What do you mean by "effectively contradicting each other". Could you please elaborate. Because the above rule set was working fine as per our requirement.

The rules mentioned above aren't effectively contradicting each other since they are just matching regular expressions on the entire list of tags. If a key specified via regex is present in the tags array, the regex should match and the rule should pass.

Our equality operator for lists actually works if ANY values in the list match, so you should be able to rewrite your above ruleset as:

AWS::EC2::VPC Tags.*.Key == Environment << the mandatory Tag: Environment is not specified
AWS::EC2::VPC Tags.*.Key == ApplicationName << the mandatory Tag: ApplicationName is not specified
AWS::EC2::VPC Tags.*.Key == SupportContact << the mandatory Tag: SupportContact is not specified
AWS::EC2::VPC Tags.*.Key == ApplicationOwner << the mandatory Tag: ApplicationOwner is not specified
AWS::EC2::VPC Tags.*.Key == CostCenter << the mandatory Tag: CostCenter is not specified

Tested with a minimal example

ruleset:

AWS::EC2::Instance Tags.*.Key == KeyValue

template:

{
    "Resources": {
        "MyResource":{
            "Type": "AWS::EC2::Instance",
            "Properties": {
                "Tags": [
                    {"Key": "MyValue"},
                    {"Key":"KeyValue"}
                ]
            }
        }
    }
}

result:

(env) 3c22fb7f6043:cfn-guard jotompki$ cfn-guard check --template template.json --rule_set ruleset.ruleset 
(env) 3c22fb7f6043:cfn-guard jotompki$ cfn-guard check --template template.json --rule_set ruleset.ruleset  -vv
2020-12-17 15:30:11,491 DEBUG [cfn_guard] Parameters are ArgMatches {
    args: {
        "template": MatchedArg {
            occurs: 1,
            indices: [
                2,
            ],
            vals: [
                "template.json",
            ],
        },
        "v": MatchedArg {
            occurs: 2,
            indices: [
                5,
                6,
            ],
            vals: [],
        },
        "rule_set": MatchedArg {
            occurs: 1,
            indices: [
                4,
            ],
            vals: [
                "ruleset.ruleset",
            ],
        },
    },
    subcommand: None,
    usage: Some(
        "USAGE:\n    cfn-guard check [FLAGS] --rule_set <RULE_SET_FILE> --template <TEMPLATE_FILE>",
    ),
}
2020-12-17 15:30:11,492 INFO  [cfn_guard] CloudFormation Guard is checking the template 'template.json' against the rules in 'ruleset.ruleset'
2020-12-17 15:30:11,492 DEBUG [cfn_guard] Entered run
2020-12-17 15:30:11,493 INFO  [cfn_guard] Loading CloudFormation Template and Rule Set
2020-12-17 15:30:11,493 DEBUG [cfn_guard] Entered run_check
2020-12-17 15:30:11,493 DEBUG [cfn_guard] Deserializing CloudFormation template
2020-12-17 15:30:11,493 INFO  [cfn_guard] Parsing rule set
2020-12-17 15:30:11,493 DEBUG [cfn_guard::parser] Entered parse_rules
2020-12-17 15:30:11,493 DEBUG [cfn_guard::parser] Parsing 'AWS::EC2::Instance Tags.*.Key == KeyValue'
2020-12-17 15:30:11,495 DEBUG [cfn_guard::parser] line_type is Rule
2020-12-17 15:30:11,495 DEBUG [cfn_guard::parser] Line is an |OR| rule
2020-12-17 15:30:11,495 DEBUG [cfn_guard::parser] Rule |OR| branch is 'AWS::EC2::Instance Tags.*.Key == KeyValue'
2020-12-17 15:30:11,496 DEBUG [cfn_guard::parser] Parsed rule is: CompoundRule(
    CompoundRule {
        compound_type: OR,
        raw_rule: "AWS::EC2::Instance Tags.*.Key == KeyValue",
        rule_list: [
            SimpleRule(
                Rule {
                    resource_type: "AWS::EC2::Instance",
                    field: "Tags.1.Key",
                    operation: Require,
                    value: "KeyValue",
                    rule_vtype: Value,
                    custom_msg: None,
                },
            ),
            SimpleRule(
                Rule {
                    resource_type: "AWS::EC2::Instance",
                    field: "Tags.0.Key",
                    operation: Require,
                    value: "KeyValue",
                    rule_vtype: Value,
                    custom_msg: None,
                },
            ),
        ],
    },
)
2020-12-17 15:30:11,496 DEBUG [cfn_guard::parser] Variables dictionary is {"ENV_HOME": "********", "ENV_USER": "********", "ENV_TERM_SESSION_ID": "********", "ENV_PWD": "********", "ENV_TERM_PROGRAM_VERSION": "********", "ENV_PS1": "********", "ENV_XPC_SERVICE_NAME": "********", "ENV_SHELL": "********", "ENV_SHLVL": "********", "ENV_HOMEBREW_GITHUB_API_TOKEN": "********", "ENV_TERM_PROGRAM": "********", "ENV_SSH_AUTH_SOCK": "********", "ENV__": "********", "ENV_TERM": "********", "ENV_OLDPWD": "********", "ENV_PATH": "********", "ENV_TMPDIR": "********", "ENV_VIRTUAL_ENV": "********", "ENV_XPC_FLAGS": "********", "ENV_LANG": "********", "ENV_LOGNAME": "********"}
2020-12-17 15:30:11,496 DEBUG [cfn_guard::parser] Rule Set is [
    CompoundRule(
        CompoundRule {
            compound_type: OR,
            raw_rule: "AWS::EC2::Instance Tags.*.Key == KeyValue",
            rule_list: [
                SimpleRule(
                    Rule {
                        resource_type: "AWS::EC2::Instance",
                        field: "Tags.1.Key",
                        operation: Require,
                        value: "KeyValue",
                        rule_vtype: Value,
                        custom_msg: None,
                    },
                ),
                SimpleRule(
                    Rule {
                        resource_type: "AWS::EC2::Instance",
                        field: "Tags.0.Key",
                        operation: Require,
                        value: "KeyValue",
                        rule_vtype: Value,
                        custom_msg: None,
                    },
                ),
            ],
        },
    ),
]
2020-12-17 15:30:11,496 INFO  [cfn_guard] Checking resources
2020-12-17 15:30:11,496 INFO  [cfn_guard] Applying rule 'CompoundRule(
    CompoundRule {
        compound_type: OR,
        raw_rule: "AWS::EC2::Instance Tags.*.Key == KeyValue",
        rule_list: [
            SimpleRule(
                Rule {
                    resource_type: "AWS::EC2::Instance",
                    field: "Tags.1.Key",
                    operation: Require,
                    value: "KeyValue",
                    rule_vtype: Value,
                    custom_msg: None,
                },
            ),
            SimpleRule(
                Rule {
                    resource_type: "AWS::EC2::Instance",
                    field: "Tags.0.Key",
                    operation: Require,
                    value: "KeyValue",
                    rule_vtype: Value,
                    custom_msg: None,
                },
            ),
        ],
    },
)'
2020-12-17 15:30:11,496 DEBUG [cfn_guard] Applying rule 'Rule { resource_type: "AWS::EC2::Instance", field: "Tags.1.Key", operation: Require, value: "KeyValue", rule_vtype: Value, custom_msg: None }'
2020-12-17 15:30:11,496 INFO  [cfn_guard] Checking [MyResource] which is of type "AWS::EC2::Instance"
2020-12-17 15:30:11,496 DEBUG [cfn_guard] Template val is String("KeyValue")
2020-12-17 15:30:11,496 DEBUG [cfn_guard] rule_val is KeyValue and val is "KeyValue"
2020-12-17 15:30:11,496 DEBUG [cfn_guard] OpCode::Require with rule_val as KeyValue and val as "KeyValue" of RValueType::Value
2020-12-17 15:30:11,496 INFO  [cfn_guard] Result: PASS
2020-12-17 15:30:11,496 DEBUG [cfn_guard] Applying rule 'Rule { resource_type: "AWS::EC2::Instance", field: "Tags.0.Key", operation: Require, value: "KeyValue", rule_vtype: Value, custom_msg: None }'
2020-12-17 15:30:11,496 INFO  [cfn_guard] Checking [MyResource] which is of type "AWS::EC2::Instance"
2020-12-17 15:30:11,497 DEBUG [cfn_guard] Template val is String("MyValue")
2020-12-17 15:30:11,497 DEBUG [cfn_guard] rule_val is KeyValue and val is "MyValue"
2020-12-17 15:30:11,497 DEBUG [cfn_guard] OpCode::Require with rule_val as KeyValue and val as "MyValue" of RValueType::Value
2020-12-17 15:30:11,497 INFO  [cfn_guard] Result: FAIL
2020-12-17 15:30:11,497 DEBUG [cfn_guard] Outcome was: '[]'
2020-12-17 15:30:11,497 INFO  [cfn_guard] All CloudFormation resources passed

[ChrisPates]

The requested functionality is equivalent to functionality in Cloud Custodian (c7n) and is highly desirable to improve developer/builder experience and push enforcement left. In line with the recommendation we give in the Tagging Best Practice whitepaper.

Currently recursively building Cfn Hooks for all resource types that support the tagris tagging standard is a lengthy (~10mins) build process, as demonstrated in this set of labs in the Tagging Workshop.

[sujit-kulkarni]

Hi,

Can we enforce the tagging at the stack-level, I tried it and it worked at the resource level only.

Thanks,