Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix for bicep issue #15458 #16237

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

fix for bicep issue #15458 #16237

wants to merge 8 commits into from

Conversation

ouldsid
Copy link
Member

@ouldsid ouldsid commented Jan 29, 2025

Contributing a Pull Request

If you haven't already, read the full contribution guide. The guide may have changed since the last time you read it, so please double-check. Once you are done and ready to submit your PR, run through the relevant checklist below.

Contributing to documentation

Contributing an example

We are integrating the Bicep examples into the Azure QuickStart Templates. If you'd like to contribute new example .bicep files that showcase abilities of the language, please follow these instructions to add them directly there. We can still take bug reports and fixes for the existing examples for the time being.

  • This is a bug fix for an existing example
  • I have resolved all warnings and errors shown by the Bicep VS Code extension
  • I have checked that all tests are passing by running dotnet test
  • I have consistent casing for all of my identifiers and am using camelCasing unless I have a justification to use another casing style

Contributing a feature

  • I have opened a new issue for the proposal, or commented on an existing one, and ensured that the Bicep maintainers are good with the design of the feature being implemented
  • I have included "Fixes #{issue_number}" in the PR description, so GitHub can link to the issue and close it when the PR is merged
  • I have appropriate test coverage of my new feature

Contributing a snippet

  • I have a snippet that is either a single, generic resource or multi resource that uses parent-child syntax

  • I have checked that there is not an equivalent snippet already submitted

  • I have used camelCasing unless I have a justification to use another casing style

  • I have placeholders values that correspond to their property names (e.g. dnsPrefix: 'dnsPrefix'), unless it's a property that MUST be changed or parameterized in order to deploy. In that case, I use 'REQUIRED' e.g. keyData

  • I have my symbolic name as the first tab stop ($1) in the snippet. e.g. res-aks-cluster.bicep

  • I have a resource name property equal to "name"

  • If applicable, I have set the location property to location: /*${<id>:location}*/'location' (not resourceGroup().location) where <id> is a placeholder id, and added param location string to the test's main.bicep file so that the resulting main.combined.bicep file used in the tests compiles without errors

  • I have verified that the snippet deploys correctly when used in the context of an actual bicep file

    e.g.

    resource aksCluster 'Microsoft.ContainerService/managedClusters@2021-03-01' = {
      name: 'name'
Microsoft Reviewers: Open in CodeFlow

Copy link
Contributor

github-actions bot commented Jan 29, 2025

Test this change out locally with the following install scripts (Action run 13190289530)

VSCode
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-vsix.sh) --run-id 13190289530
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-vsix.ps1) } -RunId 13190289530"
Azure CLI
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 13190289530
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 13190289530"

Copy link
Contributor

github-actions bot commented Jan 29, 2025

Dotnet Test Results

    78 files   -     39      78 suites   - 39   31m 51s ⏱️ - 17m 29s
11 734 tests  -      9  11 734 ✅  -      9  0 💤 ±0  0 ❌ ±0 
27 226 runs   - 13 596  27 226 ✅  - 13 596  0 💤 ±0  0 ❌ ±0 

Results for commit dea5122. ± Comparison against base commit 93c5975.

This pull request removes 1795 and adds 615 tests. Note that renamed tests count towards both.

		nestedProp1: 1
		nestedProp2: 2
		prop1: true
		prop2: false
	1
	2
	\$'")
	prop1: true
	prop2: false
…
Bicep.Cli.IntegrationTests.GenerateParamsCommandTests ‑ GenerateParams_ExplicitOutputFormatBicepParam_ExplicitIncludeParamsAll_OneParameterWithDefaultValueAsInvalidBicepSyntax_Should_Succeed
Bicep.Cli.IntegrationTests.GenerateParamsCommandTests ‑ GenerateParams_ImplicitOutputFormatJson_ExplicitIncludeParamsAll_OneArrayParameterWithDefaultValue_Should_Succeed
Bicep.Cli.IntegrationTests.GenerateParamsCommandTests ‑ GenerateParams_ImplicitOutputFormatJson_ExplicitIncludeParamsAll_OneObjectParameterWithDefaultValue_Should_Succeed
Bicep.Cli.IntegrationTests.GenerateParamsCommandTests ‑ GenerateParams_ImplicitOutputFormatJson_ExplicitIncludeParamsAll_OneParameterWithDefaultValueAsInvalidBicepSyntax_Should_Succeed
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
�ӻ
�0\u0014\u0006��>E�\u0003ĤͭBw\u0017��\u0007��\u0011+��6BA|w�A\Z\z\u0011̷��\u0004r���޴[0\u0019�
���5Ac���w��\��7��\u0012B ���;�qo��]+s���B���\u000bH�����
D�(��ʕҁ\u0012�qe�9��\u000cZrine��\u0018ؽ�\u001d�a�\u001f��F.�\	\u0016
7\u0011�J�C��Y2������PB[Aj!�Aq�:����6�\�7��<o:/�#�+\u0000\u000c\u0000\u0000,"Value cannot be null. (Parameter 'source')")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
��A\u000b�0\u0014\u0007��\u0014�\u0007��m{\u0016x\u0008:d�\u0005A�\u00189�@\u000b5\u0010��=\u000f�E�\u0016��e�7�s�?��t�2:1EI\u0015\u0013�GI�\u0018RB�~op�qxa����\u0010�\u0016���\u0005�2�^?�\u0007GWifB\u000e� \u0008\u0000\u0007
 ��	�N\u001d%>sn�:�i���^�kެ�:V��\u001f�n�\u001b\u001c�>����\u0001��\u0012'�0%\u0005�����?��Í��b\u0013-���6v�\u000e����ɲ,�\u001a�\u0013\u0015^�\u0016\u0000\u000c\u0000\u0000,"'7' is an invalid end of a number. Expected a delimiter. Path: $.INVALID_JSON | LineNumber: 0 | BytePositionInLine: 20.")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
��K
� \u0010\u0006`�=�'0N⫋��\u0015�\u0015� �Ą\u0004J��ɢЅ��4)�o�8���lo���G�\u0004��0\u0019#s�H	\u0011]\u001f�\u0004\u0002��\u0013��,\u0008���Dt��
���Y?(7Զ�ʕ�\u000bc�Ɓi�T!�\u001b�\u001d&k@o�=�������\u0012j?n��6�}�$�����D���yN@h	�Ĺ���\u000c�H&�<����\u001d$I�$kx\u0002�M��\u0000\u000c\u0000\u0000,"The path: index.json was not found in artifact contents")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003�ӽ
�0\u0010\u0007��}
� ��Ë�{ǾBh�~`,\u001a�P��C����j��
\u0019r\u001cw!�y�������i��,O9�\u001b\u0004�R��\u0003�Ȑ^�\u0001�L0���o2�k�m�*K��A�$֟��@��\u0018\u0002P\u001cs\u0014$p\u0013j��\u001a&W�E�.��9~nk74TS
k?4\u001a���ߚ�)�\u0000��"�"\u001cR���\u001a�H&�<����\u001bDQ\u0014Ekx\u0002xC ?\u0000\u000c\u0000\u0000,"The path: index.json was not found in artifact contents")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003�Խ
�0\u0010\u0007��>E�\u0003Ĥmr���Ep�\u0001b{b���V(��n:�K�K?\u0004�\u001b2�\u000er!���{�nѤX�,\u0000\u0015�\u0019\u0019\u001b�T\u0018��wD\u0008D�\u001b'\u001c�\u0002Bh;�$=�uc*;�\u001cg� _S�d9�\u0002\u0002�5p.��9��+[K�j"��i�qV�زK}+��|�}�k:\u0003>�7�\u0019��σ��?@
�.�$\Ie�Hg����A�C�m�I��\u000e�#Vކ�̵��ҳ9��8�y\u0001&�#�\u0000\u000c\u0000\u0000,"Value cannot be null. (Parameter 'source')")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003��O\u000b�0\u0014\u0000��\u0014�\u0007���\u0019x\u0008:d�\u0005A�\u00189�@\u000b5\u0010���CtQ��\u0005�w�a��{c�\u0007��nVF��� �"\u000c \u0018\u001a�\u0004c��-�$��\u0005\u0001$��\u0000x���t�W�.m+S��ADy��r\u0013aI�R\u0012!\u0006	\u0015�!6��S_\u000c��M��(+R��Ku-���/���tz��o�\u001a��\u001f\u0011\u0006�\u000e��؃2�\u0004����$3�����������z�M��g7�{r\u001c�q��\u0004���\u0000\u000c\u0000\u0000,"'7' is an invalid end of a number. Expected a delimiter. Path: $.INVALID_JSON | LineNumber: 0 | BytePositionInLine: 20.")
…

♻️ This comment has been updated with latest results.

@anthony-c-martin
Copy link
Member

@ouldsid please could you add a test to cover the scenario reported in the original issue. This is just to ensure we don't regress the behavior again in the future.

@ouldsid
Copy link
Member Author

ouldsid commented Jan 30, 2025

@ouldsid please could you add a test to cover the scenario reported in the original issue. This is just to ensure we don't regress the behavior again in the future.

There are two tests covering this scenario they did not fail because an unrelated issue was causing the test not to run (I ran into the same issue locally -> Asked Shenglong and he pointed out that he fixed it with a change that was merged yesterday I then merged with main and push a new commit (number 2 in this PR) to make sure that these test fail once this was verified I changed the test to behave as expected

Comment on lines 100 to 104
var value = PlaceholderParametersBicepParamWriter.GetValueForParameter(parameterSymbol.DeclaringParameter);

jsonWriter.WriteStartObject();
switch (parameterSymbol.Type.Name)
{
case "string":
emitter.EmitProperty("value", "");
break;
case "int":
emitter.EmitProperty("value", () => jsonWriter.WriteValue(0));
break;
case "bool":
emitter.EmitProperty("value", () => jsonWriter.WriteValue(false));
break;
case "object":
emitter.EmitProperty("value", () => { jsonWriter.WriteStartObject(); jsonWriter.WriteEndObject(); });
break;
case "array":
emitter.EmitProperty("value", () => { jsonWriter.WriteStartArray(); jsonWriter.WriteEndArray(); });
break;
}

emitter.EmitProperty("value", value);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One additional behavior change this will introduce is that a default of "" will be used for any parameter using a user-defined type if I'm reading this correctly. For example, I believe the following template:

param foo {
  requiredProperty: string
  optionalProperty: string?
}

will generate a .bicepparam file of

param foo = ''

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way to generate .bicepparam should not change. The change is to make .parameters.json consistant with it. Having said that this might be an issue but unrelated to this change

@anthony-c-martin
Copy link
Member

@ouldsid please could you add a test to cover the scenario reported in the original issue. This is just to ensure we don't regress the behavior again in the future.

There are two tests covering this scenario they did not fail because an unrelated issue was causing the test not to run (I ran into the same issue locally -> Asked Shenglong and he pointed out that he fixed it with a change that was merged yesterday I then merged with main and push a new commit (number 2 in this PR) to make sure that these test fail once this was verified I changed the test to behave as expected

Awesome, thanks!

@@ -97,25 +97,16 @@ private JToken GenerateTemplate(string contentVersion)
{
jsonWriter.WritePropertyName(parameterSymbol.Name);

var value = PlaceholderParametersBicepParamWriter.GetValueForParameter(parameterSymbol.DeclaringParameter);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't feel safe to me to depend on this method and then convert from Bicep -> JSON, because it can return invalid Bicep syntax - e.g. here:

}
return SyntaxFactory.NewlineToken;
}

My suggestion would probably be to copy the pattern used in PlaceholderParametersBicepParamWriter and introduce a dedicated method like GetValueForParameter which is able to accurately generate the right type of JSON node, including JSON comments if we're unable to generate valid syntax.

Copy link
Member Author

@ouldsid ouldsid Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is parameterDeclarationSyntax propre to bicep or can also be used by json ? my understanding is that : this.Context.SemanticModel.Root.ParameterDeclarations will be the same whether we are dealing with generating bicep or json file ...am I missing something? the value you pointed out will be handled by the the added if statement you have pointed out with your second comment

switch (parameterSymbol.Type.Name)

// emit value property only if the text is not null
if (value.ToString() != "\r\n")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is imprecise - is it possible to add logic that more specifically detects the edge cases directly against value, rather than converting to string?

I wasn't sure exactly what patterns you were aiming to detect here, so my comment is a bit vague. I can probably give you more specific suggestions if you could explain this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the case where the value is : SyntaxFactory.NewlineToken maybe I can change the if statement to be value.Type forexample?

}
else
{
if (arrayItem.Value is not null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, this is used in the compilation path of .bicep files. I'm surprised we need changes here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real issue was with the way the defaultValue is being returned. Fixed the real issue in the last iteration

switch (parameterSymbol.Type.Name)

// emit value property only if the value is not a newline token
if (value is not Token { Type: TokenType.NewLine})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NewLine

What's the significance of TokenType.NewLine for this issue? (This might be the same question as Anthony's comment.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added more context to the comment

@anthony-c-martin
Copy link
Member

anthony-c-martin commented Feb 6, 2025

I've got a general suggestion for how to solve this problem without having to deal with the complexity of converting invalid Bicep syntax to JSON.

The challenge we have is parameter default values can have functions anywhere, so you could have:

param foo object = {
  bar: 'bar'
  baz: { qux: newGuid() }
}

In which case, the .bicepparam conversion will give you:

param foo = {
  bar: 'bar'
  baz: {
    qux: ? // TODO...
  }
}

Currently you're trying to convert the value into JSON, which is extremely tricky because the compiler is intentionally not designed to compile invalid syntax. I don't see an obvious solution to this.

My suggestion is instead of trying to emit the following JSON:

{
  "parameters": {
    "foo": {
      "value": {
        "bar": "bar",
        "baz": { "qux": // TODO }
      }
  }
}

We just emit:

{
  "parameters": {
    "foo": {
      "value": // TODO
    }
  }
}

I suspect this is a much easier problem to solve because you just need to check "is the Bicep syntax valid" (which can be done with a visitor), and avoiding the conversion if it is not.

While this isn't true functional parity between .bicepparam and .json, I feel like it's "good enough" - ultimately, having functions in objects is quite an edge case, and IMO is not worth the added complexity to handle properly.

@ouldsid
Copy link
Member Author

ouldsid commented Feb 7, 2025

I've got a general suggestion for how to solve this problem without having to deal with the complexity of converting invalid Bicep syntax to JSON.

The challenge we have is parameter default values can have functions anywhere, so you could have:

param foo object = {
  bar: 'bar'
  baz: { qux: newGuid() }
}

In which case, the .bicepparam conversion will give you:

param foo = {
  bar: 'bar'
  baz: {
    qux: ? // TODO...
  }
}

Currently you're trying to convert the value into JSON, which is extremely tricky because the compiler is intentionally not designed to compile invalid syntax. I don't see an obvious solution to this.

My suggestion is instead of trying to emit the following JSON:

{
  "parameters": {
    "foo": {
      "value": {
        "bar": "bar",
        "baz": { "qux": // TODO }
      }
  }
}

We just emit:

{
  "parameters": {
    "foo": {
      "value": // TODO
    }
  }
}

I suspect this is a much easier problem to solve because you just need to check "is the Bicep syntax valid" (which can be done with a visitor), and avoiding the conversion if it is not.

While this isn't true functional parity between .bicepparam and .json, I feel like it's "good enough" - ultimately, having functions in objects is quite an edge case, and IMO is not worth the added complexity to handle properly.

Thank you @anthony-c-martin my attempt was to keep the same behavior for Json parameter writer (last iteration) when the syntax is not correct which was just to assign the value to empty string or empty object depending on the parameter type... but I think that your suggestion is much better I will implement it and send a new iteration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants