Clean docs and some rule links (#3175)

aws-cloudformation · May 3, 2024 · dfd5c73 · dfd5c73
1 parent 9c60f9f
commit dfd5c73
Show file tree

Hide file tree

Showing 7 changed files with 123 additions and 239 deletions.
diff --git a/.github/ISSUE_TEMPLATE/1.bug_report.yaml b/.github/ISSUE_TEMPLATE/1.bug_report.yaml
@@ -9,7 +9,7 @@ body:
 
         For reference, the current version of cfn-lint is <img src="https://badge.fury.io/py/cfn-lint.svg" />.
 
-        Cfn-lint uses the [CloudFormation Resource Specifications](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html) as the base to do validation. These files are included as part of the application version. Please update to the latest version of `cfn-lint` or update the spec files manually (`cfn-lint -u`)
+        Cfn-lint uses the [CloudFormation resource provider schemas](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-type-schemas.html) as the base to do validation. These files are included as part of the application version. Please update to the latest version of `cfn-lint` or update the spec files manually (`cfn-lint -u`)
   - type: input
     attributes:
       label: CloudFormation Lint Version

diff --git a/README.md b/README.md
diff --git a/docs/cfn-resource-specification.md b/docs/cfn-resource-specification.md
diff --git a/docs/cfn-schema-specification.md b/docs/cfn-schema-specification.md
@@ -38,32 +38,52 @@ Resource types may have complex rules to define what a valid resource configurat
 
 To make schema writing easier across hundereds of resources we have extend the schemas to include some additional keywords. While these keywords can be covered under the JSON schema they have to be done with a combination of `if`s, `onlyOne`s, `anyOf`s, etc. By using these keywords we can extend the schema for common scenarios when writing CloudFormation schemas.
 
-#### pattern
+#### type
 
-_pattern_ keyword is used to validate a string against a regular expression. [JSON Schema docs](https://json-schema.org/understanding-json-schema/reference/string#regexp)
+_type_ specifies the data type for a schema. [JSON Schema docs](https://json-schema.org/understanding-json-schema/reference/type)
 
 #### enum
 
 _enum_ is used to restrict a value to a fixed set of values. [JSON Schema docs](https://json-schema.org/understanding-json-schema/reference/string#enum)
 
-#### number range
+#### Strings
+
+##### pattern
+
+_pattern_ keyword is used to validate a string against a regular expression. [JSON Schema docs](https://json-schema.org/understanding-json-schema/reference/string#regexp)
+
+#### length
+
+_minLength_ and _maxLength_ are used to are used to constrain the size of a string. [JSON Schema docs](https://json-schema.org/understanding-json-schema/reference/string#length)
+
+#### Numbers or Integers
+
+##### number range
 
 _minimum_ and _maximum_ is used to define the inclusive range for a number or integer.
 _exclusiveMinimum_ and _exclusiveMaximum_ is used to define the exlusive range for a number or integer.
 
-#### array length
+#### Arrays
+
+##### array length
 
 _minItems_ and _maxItems_ is used to provide the inclusive length of an array.
 
-#### properties
+##### prefixItems
+
+_prefixItems_ is similar to the definition of [prefixItems](https://json-schema.org/understanding-json-schema/reference/array#tupleValidation) but doesn't actually do the prefix. The current resource schema doesn't support [items](https://json-schema.org/understanding-json-schema/reference/array#items) being an array. We use `prefixItems` to validate array items where ordering matters.
+
+#### Objects
+
+##### properties
 
 _properties_ provides the key names and a value that represents the schema to validate the property for an object. [JSON Schema Docs](https://json-schema.org/understanding-json-schema/reference/object#properties)
 
-#### required
+##### required
 
 _required_ defines a list of required properties. [JSON Schema docs](https://json-schema.org/understanding-json-schema/reference/object#required)
 
-#### requiredXor
+##### requiredXor
 
 _requiredXor_ is used to define when only one property from a set properties is required.
 
@@ -106,7 +126,7 @@ is equivalent to the JSON schema
 }
 ```
 
-#### propertiesNand
+##### propertiesNand
 
 _propertiesNand_ is used to define when none or only one property from a set properties can be defined.
 
@@ -156,11 +176,11 @@ is equivalent to the JSON schema
 }
 ```
 
-#### dependentRequired
+##### dependentRequired
 
 _dependentRequired_ has been backported into cfn-lint. You can read the definition [here](https://json-schema.org/understanding-json-schema/reference/conditionals#dependentRequired)
 
-#### dependentExcluded
+##### dependentExcluded
 
 _dependentExcluded_ is the opposite of dependentRequired. The list of properties should not be specified when the key property is specified.
 
@@ -201,11 +221,3 @@ is equivalent to the JSON schema
   }
 }
 ```
-
-#### prefixItems
-
-_prefixItems_ is similar to the definition of [prefixItems](https://json-schema.org/understanding-json-schema/reference/array#tupleValidation) but doesn't actually do the prefix. The current resource schema doesn't support [items](https://json-schema.org/understanding-json-schema/reference/array#items) being an array. We use `prefixItems` to validate array items where ordering matters.
-
-#### type
-
-_type_ specifies the data type for a schema. [JSON Schema docs](https://json-schema.org/understanding-json-schema/reference/type)
diff --git a/docs/internal.md b/docs/internal.md
@@ -34,139 +34,28 @@ cfn-lint --update-documentation
 
 The official resource specifications are one source of data, the other two are the "extended specs" which are "patches" to the spec that enforce more constraints, and the "additional specs" which are rules written in JSON format that are then picked up by their respective Python class.
 
-#### CloudSpecs
-
-The command `cfn-lint --update-specs` pulls down the official resource specifications into folder `CloudSpecs` and patches the JSON files with the contents of the files in `ExtendedSpecs`. The merged results are stored in `CloudSpecs`.
-
-#### ExtendedSpecs
-
-These files follow the [JsonPatch format](http://jsonpatch.com/) and are merged with the official specs. They support the following syntax:
-
-- Allowed patterns. [Example](https://github.com/aws-cloudformation/cfn-python-lint/blob/a46773c752247c51effef415bd3462eaec10ab0b/src/cfnlint/data/ExtendedSpecs/all/03_value_types.json#L33-L36):
-
-    ```json
-    {
-        "op": "add",
-        "path": "/ValueTypes",
-        "value": {
-            "CidrIp": {
-                "AllowedPattern": "x.x.x.x/y",
-                "AllowedPatternRegex": "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$"
-            }
-        }
-    }
-    ```
-
-- Allowed values. One example: [`ExtendedSpecs/$REGION/05_pricing_property_values.json`](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/data/ExtendedSpecs/all/05_pricing_property_values.json) validates EMR instance types and is generated by [`scripts/update_specs_from_pricing.py`](https://github.com/aws-cloudformation/cfn-python-lint/blob/cc6ac28ff7deba86cb82813733cceec4bdff68a2/scripts/update_specs_from_pricing.py#L235). [Another example](https://github.com/aws-cloudformation/cfn-python-lint/blob/6cce9222c89056f1546f6fee068ce6dc9dfa394e/src/cfnlint/data/ExtendedSpecs/all/03_value_types/aws_codebuild.json#L2-L11):
-
-    ```json
-    {
-        "op": "add",
-        "path": "/ValueTypes/AWS::CodeBuild::Project.Artifacts.Packaging",
-        "value": {
-            "AllowedValues": [
-                "NONE",
-                "ZIP"
-            ]
-        }
-    }
-    ```
-
-- List size constraints. [Example](https://github.com/aws-cloudformation/cfn-python-lint/blob/6cce9222c89056f1546f6fee068ce6dc9dfa394e/src/cfnlint/data/ExtendedSpecs/all/03_value_types/aws_iam.json#L71-L78):
-
-    ```json
-    {
-        "op": "add",
-        "path": "/ValueTypes/AWS::IAM::Group.Names",
-        "value": {
-            "ListMax": 10,
-            "ListMin": 0
-        }
-    }
-    ```
-
-- Number size constraints. [Example](https://github.com/aws-cloudformation/cfn-python-lint/blob/df8d065380e49e53dad9513dab41c2438e105f43/src/cfnlint/data/ExtendedSpecs/all/03_value_types/aws_sqs.json#L17-L24):
-
-    ```json
-    {
-        "op": "add",
-        "path": "/ValueTypes/AWS::SQS::Queue.MaximumMessageSize",
-        "value": {
-            "NumberMax": 262144,
-            "NumberMin": 1024
-        }
-    }
-    ```
-
-- String size constraints. [Example](https://github.com/aws-cloudformation/cfn-python-lint/blob/67fc5bb210b020e3226261f966a01726d574475d/src/cfnlint/data/ExtendedSpecs/all/03_value_types/aws_logs.json#L2-L9):
-
-    ```json
-    {
-        "op": "add",
-        "path": "/ValueTypes/AWS::Logs::LogGroup.LogGroupName",
-        "value": {
-            "StringMax": 512,
-            "StringMin": 1
-        }
-    }
-    ```
-
-There should be no functional difference, but [`src/cfnlint/data/ExtendedSpecs/all/03_value_types`](https://github.com/aws-cloudformation/cfn-python-lint/tree/main/src/cfnlint/data/ExtendedSpecs/all/03_value_types) and [`src/cfnlint/data/ExtendedSpecs/all/04_property_values`](https://github.com/aws-cloudformation/cfn-python-lint/tree/main/src/cfnlint/data/ExtendedSpecs/all/04_property_values) are more organized than [`src/cfnlint/data/ExtendedSpecs/all/03_value_types.json`](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/data/ExtendedSpecs/all/03_value_types.json) and [`src/cfnlint/data/ExtendedSpecs/all/04_property_values.json`](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/data/ExtendedSpecs/all/04_property_values.json), so they should be preferred locations for new constraints.
-
-[`ExtendedSpecs/$REGION/06_ssm_service_removal.json`](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/data/ExtendedSpecs/us-gov-east-1/06_ssm_service_removal.json) is written by [`scripts/update_specs_services_from_ssm.py`](https://github.com/aws-cloudformation/cfn-python-lint/blob/cc6ac28ff7deba86cb82813733cceec4bdff68a2/scripts/update_specs_services_from_ssm.py#L165) and [`ExtendedSpecs/$REGION/07_ssm_service_addition.json`](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/data/ExtendedSpecs/us-gov-east-1/07_ssm_service_addition.json) is written by [`scripts/update_specs_services_from_ssm.py`](https://github.com/aws-cloudformation/cfn-python-lint/blob/cc6ac28ff7deba86cb82813733cceec4bdff68a2/scripts/update_specs_services_from_ssm.py#L204).
-
-If we push changes to these files, customers will have to update `cfn-lint`. The person changing the file(s) can also see the changes by running the following:
+#### Schemas
 
-```shell
-pip3 install -e .
-cfn-lint --update-specs # https://github.com/aws-cloudformation/cfn-python-lint/pull/1383#issuecomment-629891506
-```
+The command `cfn-lint --update-specs` pulls down the official resource specifications into folder `schemas` and patches the JSON files with the contents of the files in `patches` folder. The merged results are stored in `providers`.
+
+##### Extensions
+
+Extensions are used to extend the provider schemas. We use these schemas for specific tests where it can be hard to nest it in the resource provider schema. Using extensions allow us to create separate rule IDs for each extension which allows the customer to ignore the error or for us to change the rule level (example: warning)
+
+##### Other
+
+The other folder has any schema used for validation that isn't under a resources properties. This includes schemas for the overall template structure of a CloudFormation template, IAM policy schemas, CFN Init schemas, and more.
+
+##### Patches
+
+Patches contain all the patches we apply to the provider schemas when they are downloaded. There are two folders inside patches. _providers_ patch issues in the provider schema itself and _extensions_ apply additions to the provider schema to create better linting results.
+
+##### Providers
+
+Providers stores all the regional resource provider schemas after they are patched. Files are deduplicated so that we are storing as little as possible. If you look at the `__init__.py` file you will see what resources are cached.
 
 #### AdditionalSpecs
 
 If we push changes to these files, customers will have to update their version of `cfn-lint`. They support the following syntax:
 
-- [At least one of these properties must be specified](https://github.com/aws-cloudformation/cfn-python-lint/blob/b788cc9bd3d49ed20d5f2e58602755a0ef37f52c/src/cfnlint/data/AdditionalSpecs/AtLeastOne.json#L20-L25), used by rule [E2522](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/rules/resources/properties/AtLeastOne.py). Example:
-
-    ```json
-    "AWS::EC2::Instance": [
-        [
-            "ImageId",
-            "LaunchTemplate"
-        ]
-    ]
-    ```
-
-- [Only one of these properties may be specified](https://github.com/aws-cloudformation/cfn-python-lint/blob/b788cc9bd3d49ed20d5f2e58602755a0ef37f52c/src/cfnlint/data/AdditionalSpecs/OnlyOne.json#L79-L84), used by rule [E2523](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/rules/resources/properties/OnlyOne.py). Example:
-
-    ```json
-    "AWS::CloudWatch::Alarm": [
-        [
-            "MetricName",
-            "Metrics"
-        ]
-    ]
-    ```
-
-- [If this property is specified, these properties must be excluded](https://github.com/aws-cloudformation/cfn-python-lint/blob/b788cc9bd3d49ed20d5f2e58602755a0ef37f52c/src/cfnlint/data/AdditionalSpecs/Exclusive.json#L102-L107), used by rule [E2520](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/rules/resources/properties/Exclusive.py). Example:
-
-    ```json
-    "AWS::RDS::DBCluster": {
-        "SnapshotIdentifier": [
-            "MasterUsername",
-            "MasterUserPassword"
-        ]
-    }
-    ```
-
-- [If this property is specified, these properties must be included](https://github.com/aws-cloudformation/cfn-python-lint/blob/b788cc9bd3d49ed20d5f2e58602755a0ef37f52c/src/cfnlint/data/AdditionalSpecs/Inclusive.json#L48-L52), used by rule [E2521](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/rules/resources/properties/Inclusive.py). Example:
-
-    ```json
-    "AWS::OpsWorks::Stack": {
-        "VpcId": [
-            "DefaultSubnetId"
-        ]
-    }
-    ```
-
 [`AdditionalSpecs/RdsProperties.json`](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/data/AdditionalSpecs/RdsProperties.json) is written by [`scripts/update_specs_from_pricing.py`](https://github.com/aws-cloudformation/cfn-python-lint/blob/cc6ac28ff7deba86cb82813733cceec4bdff68a2/scripts/update_specs_from_pricing.py#L189) and used by rule [E3025](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/rules/resources/rds/InstanceSize.py) and [`AdditionalSpecs/Policies.json`](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/data/AdditionalSpecs/Policies.json) is written by [`cfn-lint --update-iam-policies`](https://github.com/aws-cloudformation/cfn-python-lint/blob/cc6ac28ff7deba86cb82813733cceec4bdff68a2/src/cfnlint/maintenance.py#L173) and used by rule [W3037](https://github.com/aws-cloudformation/cfn-python-lint/blob/main/src/cfnlint/rules/resources/iam/Permissions.py).
diff --git a/src/cfnlint/rules/resources/properties/NumberRange.py b/src/cfnlint/rules/resources/properties/NumberRange.py
@@ -21,7 +21,7 @@ class NumberRange(CloudFormationLintRule):
         "Check numbers (integers and floats) for its value being between the minimum"
         " and maximum"
     )
-    source_url = "https://github.com/awslabs/cfn-python-lint/blob/main/docs/cfn-resource-specification.md#allowedpattern"
+    source_url = "https://github.com/aws-cloudformation/cfn-lint/blob/main/docs/cfn-schema-specification.md#number-range"
     tags = ["resources", "property", "number", "size"]
     child_rules = {
         "W3034": None,

diff --git a/src/cfnlint/rules/resources/properties/StringLength.py b/src/cfnlint/rules/resources/properties/StringLength.py
@@ -20,7 +20,7 @@ class StringLength(CloudFormationLintRule):
     id = "E3033"
     shortdesc = "Check if a string has between min and max number of values specified"
     description = "Check strings for its length between the minimum and maximum"
-    source_url = "https://github.com/awslabs/cfn-python-lint/blob/main/docs/cfn-resource-specification.md#allowedpattern"
+    source_url = "https://github.com/aws-cloudformation/cfn-lint/blob/v1/docs/cfn-schema-specification.md#length"
     tags = ["resources", "property", "string", "size"]
 
     def _serialize_date(self, obj):