[Configuration] Initialize optional properties with defaults #1684
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Why make this change? - With the overhaul of the config system using our own Write method while serializing objects, the config that is generated is misaligned with respect to indentation for all the keys under each entity name. For e.g.: In the following picture, the entity name `Todo` is indented with 4 leading spaces, subsequently its child key `source` should have been indented with 6 leading spaces, however its currently indented with only 2 spaces.  - This is a regression in 0.8.44-rc. ## What is this change? - Use the same writer to serialize the subkeys under the entity name so that the depth of indentation is maintained. ## How does this change fix the issue? - `WriteRawValue` writes the json string argument provided as is without doing any additional formatting on it. It doesn't utilize the `_currentDepth` private member of the writer to determine how to indent. - Prior to this change, we were generating the inner json string first then writing the generated inner json "as is". While generating that json on line 29 https://github.com/Azure/data-api-builder/blob/31b754251832ffebf4c4ce36ffcbead9baff6355/src/Config/Converters/RuntimeEntitiesConverter.cs#L29 Line 29 in [31b7542](31b7542) a new writer is being used by the `JsonSerializer.Serialize()` function, effectively resetting the `_currentDepth`. The `_currentDepth` of the writer in context was not getting honored. ## How was this tested? - Manual test, building solution and using `dab add` to simulate automatic writing of an entity to the config file. With the change, the indentation is fixed. 
… into release/0.8
Aniruddh25
requested review from
mbhaskar,
JelteF,
seantleonard,
Mathos1432,
gledis69,
aaronburtle,
milismsft,
tarazou9,
ayush3797,
abhishekkumams,
aaronpowell,
severussundar and
ravishetye
as code owners
Aniruddh25
changed the title
Aniruddh25
changed the title
abhishekkumams
approved these changes
Sep 5, 2023
ayush3797
approved these changes
Sep 5, 2023
Aniruddh25
added a commit
that referenced
this pull request
Sep 19, 2023
## Why make this change? - As per the JSON config schema, not all properties in the config file are mandatory. - With #1402, in versions 0.8.49+ those properties that are not explicitly set, are initialized with defaults before writing to the file system. - However, config files generated using version 0.7.6 or lesser, may not have these properties initialized to their defaults when writing to the file system. Using config files generated with <= 0.7.6, to start engine with 0.8.49 would therefore lead to null reference exceptions when the optional properties are dereferenced. - This PR is to ensure such null reference exceptions are avoided when starting engine with configs that don't have optional properties ## What is this change? - Modify object model constructors to accept nullable arguments for optional properties - represented as fields in the record types. If passed in argument is null, initialize the field with default values. Do this recursively for sub-properties. - To force JSON deserializer to pick up the custom constructor, add `IncludeFields = true` to `JsonSerializerOptions` - Default value of host mode is changed to `development` since `allow-introspection` has a default value of `true` ## How was this tested? - Removed optional properties from config and ensured engine could be started without them. - [X] Unit Tests ## Sample Request(s) - remove `runtime` section from config, starting engine with 0.8.49 leads to NullReferenceException. - After the fix, starting engine is successful even when `runtime` section is absent in the config.
2 tasks
2 tasks
Aniruddh25
added a commit
that referenced
this pull request
Oct 20, 2023
## Why make this change? - As per the JSON config schema, not all properties in the config file are mandatory. - With #1402, in versions 0.8.49-0.8.52 those properties that are not explicitly set, were initialized with defaults before writing to the file system. - However, config files generated using version 0.7.6 or lesser, may not have these properties initialized to their defaults when writing to the file system. Using config files generated with <= 0.7.6, to start engine with 0.8.49 would therefore lead to null reference exceptions when the optional properties are dereferenced. - With PR #1684 the optional properties were initialized with their default values to avoid the null reference, any config file that is created or modified using the CLI results in a fully expanded file. This is not a problem when a single config is used but leads to undesired behavior when using the merge configuration file feature. A fully expanded higher precedence config file when merged with a base config file will override the values set for the optional properties in the base file with their default values if those properties were not explicitly defined in the higher precedence config file to begin with. In order to retain the base config file values for the optional properties, the higher precedence file would have to define every single optional value exactly the same as in the base file if they desire those to not be overridden. That defeats the purpose of providing the higher precedence file which should ideally only have those properties that need to be overriden. For more details with examples, please refer to #1737. - This PR fixes this problem by allowing optional properties to be null and ensures there are no null reference exceptions as well when starting engine with configs that don't have optional properties. ## What is this change? - Modify object model constructors to accept nullable arguments for optional properties - represented as fields in the record types. The scalar optional properties have not been made nullable yet, will have a follow up PR for those. - All references to optional properties check for nulls - During merge of config files, null + non-null values will pick up non-null values. After the merge, if the value is still null, they are considered as default values. To easily get default values, added some properties to `RuntimeConfig`. - Default value of Host.Mode is `Production`. Keep it that way. Default of GraphQL.AllowIntrospection is true currently - will have another PR to determine it based on host mode. ## How was this tested? - Removed optional properties from config and ensured engine could be started without them. - [X] Unit Tests for deserialization and serialization of nullable optional properties see `RuntimeConfigLoaderJsonDeserializerTests` - [X] In the merge configuration tests, in `TestHelper`, modify base config to contain a non-default value, env based config to not specify that value and confirm the merged config file has the base-config value. ## Sample Request(s) - Before fix, remove `runtime` section from config, run dab start with 0.8.49 leads to NullReferenceException. - After the fix, starting engine is successful even when `runtime` section is absent in the config. // IN THE BASE config file MISSING = use default { empty } = use default // IN THE OVERRIDDEN config file MISSING = use base { empty } = use base Examples: // all subproperties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { empty } merged: "user": { "first": "jerry", "last": "nixon" } // some sub properties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { "first": "jerry" } merged: "user": { "first": "jerry", "last": "nixon" } // parent object property is missing altogether base: "user": { "first": "jerry", "last": "nixon" } override: "user": MISSING (not specified in override) merged: "user": { "first": "jerry", "last": "nixon" } ## TO DO for 0.10-rc - A followup PR to make scalar optional properties nullable too. - A followup PR that makes specifying an explicit NULL imply Default values
Aniruddh25
added a commit
that referenced
this pull request
Oct 20, 2023
## Why make this change? - As per the JSON config schema, not all properties in the config file are mandatory. - With #1402, in versions 0.8.49-0.8.52 those properties that are not explicitly set, were initialized with defaults before writing to the file system. - However, config files generated using version 0.7.6 or lesser, may not have these properties initialized to their defaults when writing to the file system. Using config files generated with <= 0.7.6, to start engine with 0.8.49 would therefore lead to null reference exceptions when the optional properties are dereferenced. - With PR #1684 the optional properties were initialized with their default values to avoid the null reference, any config file that is created or modified using the CLI results in a fully expanded file. This is not a problem when a single config is used but leads to undesired behavior when using the merge configuration file feature. A fully expanded higher precedence config file when merged with a base config file will override the values set for the optional properties in the base file with their default values if those properties were not explicitly defined in the higher precedence config file to begin with. In order to retain the base config file values for the optional properties, the higher precedence file would have to define every single optional value exactly the same as in the base file if they desire those to not be overridden. That defeats the purpose of providing the higher precedence file which should ideally only have those properties that need to be overriden. For more details with examples, please refer to #1737. - This PR fixes this problem by allowing optional properties to be null and ensures there are no null reference exceptions as well when starting engine with configs that don't have optional properties. ## What is this change? - Modify object model constructors to accept nullable arguments for optional properties - represented as fields in the record types. The scalar optional properties have not been made nullable yet, will have a follow up PR for those. - All references to optional properties check for nulls - During merge of config files, null + non-null values will pick up non-null values. After the merge, if the value is still null, they are considered as default values. To easily get default values, added some properties to `RuntimeConfig`. - Default value of Host.Mode is `Production`. Keep it that way. Default of GraphQL.AllowIntrospection is true currently - will have another PR to determine it based on host mode. ## How was this tested? - Removed optional properties from config and ensured engine could be started without them. - [X] Unit Tests for deserialization and serialization of nullable optional properties see `RuntimeConfigLoaderJsonDeserializerTests` - [X] In the merge configuration tests, in `TestHelper`, modify base config to contain a non-default value, env based config to not specify that value and confirm the merged config file has the base-config value. ## Sample Request(s) - Before fix, remove `runtime` section from config, run dab start with 0.8.49 leads to NullReferenceException. - After the fix, starting engine is successful even when `runtime` section is absent in the config. // IN THE BASE config file MISSING = use default { empty } = use default // IN THE OVERRIDDEN config file MISSING = use base { empty } = use base Examples: // all subproperties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { empty } merged: "user": { "first": "jerry", "last": "nixon" } // some sub properties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { "first": "jerry" } merged: "user": { "first": "jerry", "last": "nixon" } // parent object property is missing altogether base: "user": { "first": "jerry", "last": "nixon" } override: "user": MISSING (not specified in override) merged: "user": { "first": "jerry", "last": "nixon" } ## TO DO for 0.10-rc - A followup PR to make scalar optional properties nullable too. - A followup PR that makes specifying an explicit NULL imply Default values
seantleonard
pushed a commit
that referenced
this pull request
Oct 21, 2023
## Why make this change? - As per the JSON config schema, not all properties in the config file are mandatory. - With #1402, in versions 0.8.49-0.8.52 those properties that are not explicitly set, were initialized with defaults before writing to the file system. - However, config files generated using version 0.7.6 or lesser, may not have these properties initialized to their defaults when writing to the file system. Using config files generated with <= 0.7.6, to start engine with 0.8.49 would therefore lead to null reference exceptions when the optional properties are dereferenced. - With PR #1684 the optional properties were initialized with their default values to avoid the null reference, any config file that is created or modified using the CLI results in a fully expanded file. This is not a problem when a single config is used but leads to undesired behavior when using the merge configuration file feature. A fully expanded higher precedence config file when merged with a base config file will override the values set for the optional properties in the base file with their default values if those properties were not explicitly defined in the higher precedence config file to begin with. In order to retain the base config file values for the optional properties, the higher precedence file would have to define every single optional value exactly the same as in the base file if they desire those to not be overridden. That defeats the purpose of providing the higher precedence file which should ideally only have those properties that need to be overriden. For more details with examples, please refer to #1737. - This PR fixes this problem by allowing optional properties to be null and ensures there are no null reference exceptions as well when starting engine with configs that don't have optional properties. ## What is this change? - Modify object model constructors to accept nullable arguments for optional properties - represented as fields in the record types. The scalar optional properties have not been made nullable yet, will have a follow up PR for those. - All references to optional properties check for nulls - During merge of config files, null + non-null values will pick up non-null values. After the merge, if the value is still null, they are considered as default values. To easily get default values, added some properties to `RuntimeConfig`. - Default value of Host.Mode is `Production`. Keep it that way. Default of GraphQL.AllowIntrospection is true currently - will have another PR to determine it based on host mode. ## How was this tested? - Removed optional properties from config and ensured engine could be started without them. - [X] Unit Tests for deserialization and serialization of nullable optional properties see `RuntimeConfigLoaderJsonDeserializerTests` - [X] In the merge configuration tests, in `TestHelper`, modify base config to contain a non-default value, env based config to not specify that value and confirm the merged config file has the base-config value. ## Sample Request(s) - Before fix, remove `runtime` section from config, run dab start with 0.8.49 leads to NullReferenceException. - After the fix, starting engine is successful even when `runtime` section is absent in the config. // IN THE BASE config file MISSING = use default { empty } = use default // IN THE OVERRIDDEN config file MISSING = use base { empty } = use base Examples: // all subproperties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { empty } merged: "user": { "first": "jerry", "last": "nixon" } // some sub properties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { "first": "jerry" } merged: "user": { "first": "jerry", "last": "nixon" } // parent object property is missing altogether base: "user": { "first": "jerry", "last": "nixon" } override: "user": MISSING (not specified in override) merged: "user": { "first": "jerry", "last": "nixon" } ## TO DO for 0.10-rc - A followup PR to make scalar optional properties nullable too. - A followup PR that makes specifying an explicit NULL imply Default values
2 tasks
seantleonard
pushed a commit
that referenced
this pull request
Oct 23, 2023
## Why make this change? - As per the JSON config schema, not all properties in the config file are mandatory. - With #1402, in versions 0.8.49-0.8.52 those properties that are not explicitly set, were initialized with defaults before writing to the file system. - However, config files generated using version 0.7.6 or lesser, may not have these properties initialized to their defaults when writing to the file system. Using config files generated with <= 0.7.6, to start engine with 0.8.49 would therefore lead to null reference exceptions when the optional properties are dereferenced. - With PR #1684 the optional properties were initialized with their default values to avoid the null reference, any config file that is created or modified using the CLI results in a fully expanded file. This is not a problem when a single config is used but leads to undesired behavior when using the merge configuration file feature. A fully expanded higher precedence config file when merged with a base config file will override the values set for the optional properties in the base file with their default values if those properties were not explicitly defined in the higher precedence config file to begin with. In order to retain the base config file values for the optional properties, the higher precedence file would have to define every single optional value exactly the same as in the base file if they desire those to not be overridden. That defeats the purpose of providing the higher precedence file which should ideally only have those properties that need to be overriden. For more details with examples, please refer to #1737. - This PR fixes this problem by allowing optional properties to be null and ensures there are no null reference exceptions as well when starting engine with configs that don't have optional properties. ## What is this change? - Modify object model constructors to accept nullable arguments for optional properties - represented as fields in the record types. The scalar optional properties have not been made nullable yet, will have a follow up PR for those. - All references to optional properties check for nulls - During merge of config files, null + non-null values will pick up non-null values. After the merge, if the value is still null, they are considered as default values. To easily get default values, added some properties to `RuntimeConfig`. - Default value of Host.Mode is `Production`. Keep it that way. Default of GraphQL.AllowIntrospection is true currently - will have another PR to determine it based on host mode. ## How was this tested? - Removed optional properties from config and ensured engine could be started without them. - [X] Unit Tests for deserialization and serialization of nullable optional properties see `RuntimeConfigLoaderJsonDeserializerTests` - [X] In the merge configuration tests, in `TestHelper`, modify base config to contain a non-default value, env based config to not specify that value and confirm the merged config file has the base-config value. ## Sample Request(s) - Before fix, remove `runtime` section from config, run dab start with 0.8.49 leads to NullReferenceException. - After the fix, starting engine is successful even when `runtime` section is absent in the config. // IN THE BASE config file MISSING = use default { empty } = use default // IN THE OVERRIDDEN config file MISSING = use base { empty } = use base Examples: // all subproperties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { empty } merged: "user": { "first": "jerry", "last": "nixon" } // some sub properties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { "first": "jerry" } merged: "user": { "first": "jerry", "last": "nixon" } // parent object property is missing altogether base: "user": { "first": "jerry", "last": "nixon" } override: "user": MISSING (not specified in override) merged: "user": { "first": "jerry", "last": "nixon" } ## TO DO for 0.10-rc - A followup PR to make scalar optional properties nullable too. - A followup PR that makes specifying an explicit NULL imply Default values
2 tasks
seantleonard
added a commit
that referenced
this pull request
Oct 24, 2023
Cherry pick #1823 to 0.9 branch. ## Why make this change? - As per the JSON config schema, not all properties in the config file are mandatory. - With #1402, in versions 0.8.49-0.8.52 those properties that are not explicitly set, were initialized with defaults before writing to the file system. - However, config files generated using version 0.7.6 or lesser, may not have these properties initialized to their defaults when writing to the file system. Using config files generated with <= 0.7.6, to start engine with 0.8.49 would therefore lead to null reference exceptions when the optional properties are dereferenced. - With PR #1684 the optional properties were initialized with their default values to avoid the null reference, any config file that is created or modified using the CLI results in a fully expanded file. This is not a problem when a single config is used but leads to undesired behavior when using the merge configuration file feature. A fully expanded higher precedence config file when merged with a base config file will override the values set for the optional properties in the base file with their default values if those properties were not explicitly defined in the higher precedence config file to begin with. In order to retain the base config file values for the optional properties, the higher precedence file would have to define every single optional value exactly the same as in the base file if they desire those to not be overridden. That defeats the purpose of providing the higher precedence file which should ideally only have those properties that need to be overriden. For more details with examples, please refer to #1737. - This PR fixes this problem by allowing optional properties to be null and ensures there are no null reference exceptions as well when starting engine with configs that don't have optional properties. ## What is this change? - Modify object model constructors to accept nullable arguments for optional properties - represented as fields in the record types. The scalar optional properties have not been made nullable yet, will have a follow up PR for those. - All references to optional properties check for nulls - During merge of config files, null + non-null values will pick up non-null values. After the merge, if the value is still null, they are considered as default values. To easily get default values, added some properties to `RuntimeConfig`. - Default value of Host.Mode is `Production`. Keep it that way. Default of GraphQL.AllowIntrospection is true currently - will have another PR to determine it based on host mode. ## How was this tested? - Removed optional properties from config and ensured engine could be started without them. - [X] Unit Tests for deserialization and serialization of nullable optional properties see `RuntimeConfigLoaderJsonDeserializerTests` - [X] In the merge configuration tests, in `TestHelper`, modify base config to contain a non-default value, env based config to not specify that value and confirm the merged config file has the base-config value. ## Sample Request(s) - Before fix, remove `runtime` section from config, run dab start with 0.8.49 leads to NullReferenceException. - After the fix, starting engine is successful even when `runtime` section is absent in the config. // IN THE BASE config file MISSING = use default { empty } = use default // IN THE OVERRIDDEN config file MISSING = use base { empty } = use base Examples: // all subproperties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { empty } merged: "user": { "first": "jerry", "last": "nixon" } // some sub properties are missing base: "user": { "first": "jerry", "last": "nixon" } override: "user": { "first": "jerry" } merged: "user": { "first": "jerry", "last": "nixon" } // parent object property is missing altogether base: "user": { "first": "jerry", "last": "nixon" } override: "user": MISSING (not specified in override) merged: "user": { "first": "jerry", "last": "nixon" } ## TO DO for 0.10-rc - A followup PR to make scalar optional properties nullable too. - A followup PR that makes specifying an explicit NULL imply Default values --------- Co-authored-by: Aniruddh Munde <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why make this change?
What is this change?
IncludeFields = truetoJsonSerializerOptionsdevelopmentsinceallow-introspectionhas a default value oftrueHow was this tested?
Sample Request(s)
runtimesection from config, starting engine with 0.8.49 leads to NullReferenceException.runtimesection is absent in the config.