Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADF artifact doesn't contain the global parameters per enviroment #490

Open
sofiac9 opened this issue Dec 15, 2022 · 5 comments
Open

ADF artifact doesn't contain the global parameters per enviroment #490

sofiac9 opened this issue Dec 15, 2022 · 5 comments

Comments

@sofiac9
Copy link

sofiac9 commented Dec 15, 2022

Hello,

I'm following the Global Parameters documentation. Specifically the CI/CD deployment with PowerShell. I need to deploy in a new environment with different global params values but in the documentation, it is not clear where I have to save the copy of the global parameters files with the values of the other environment. I need the artifact with two global parameters files like the documentation shows:
image

But when the process creates the artifact I only can see the parameters of one environment:
image

I save a copy of the new environment global parameters (JSON file) in the factory folder:
image

I tried with and without the arm-template-parameters-definition.json file. Also in ADF I marked the included global parameters in ARM template:

image

It is not clear to me what am I doing wrong. Am I saving the new environments JSON file in the wrong folder? The structure of the JSON file is the same as the original one but with different values.

I'm not using the new mechanism since I have a lot of global parameters in ADF. I need to use the old mechanism with PowerShell.

Thanks for your help.

@soma-ms
Copy link
Contributor

soma-ms commented Dec 21, 2022

@sofiac9 - As you want to use old mechanism to update global parameters using PowerShell script, you need to have the global-parameter json file for each of the environment in globalParameters folder in publish branch where the arm-templates/powershell scripts are present.

The checkbox "Include global parameters in ARM template" is used for new mechanism to include global parameter in arm-template. It should be unselected if you want to use old mechanism as they will overwrite one another.

@sofiac9
Copy link
Author

sofiac9 commented Dec 27, 2022

But what do I have to do in order to get the two files in the artifact?
When the CI pipeline generates the artifact I only see the development parameters.
image

  • I did it with the global parameters checkbox unselected

@soma-ms
Copy link
Contributor

soma-ms commented Dec 27, 2022

My earlier comment is applicable if you are using publish branch to deploy the factory resources to different environment. But you latest comment indicates you are using automated publish (npm utility) to generate the arm-template. In that case you have to add the _globalparameters.json file yourself to any ADO branch and configure that branch as release artifact (Source type: Azure repos). So the release pipeline can download these files to agent machine and the PS script (GlobalParameterUpdateScript.ps1) can be used to deploy global parameters.

@jontreynes
Copy link

@soma-ms - are we required to use this powershell script method to hydrate environment specfic global parameters from one ADF environment to another if not using the deprecated Include in ARM Template option?

@jontreynes
Copy link

jontreynes commented Feb 1, 2023

Update - this works for me - (However, I'm not clear on if this is as intended by the product)

"includeGlobalParamsTemplate":false under publish_config.json

For me, I have three different ADF's (let's call them dev, qa, prod). In my repo, I included dev_GlobalParameters.json, qa__GlobalParameters.json, and prod__GlobalParameters.json

in each *_GlobalParameters.json, I included the same parameters with environment specific values

When using the script under the docs, I inject the parameter, globalParametersFilePath, with the values to the path dev_GlobalParameters.json, qa__GlobalParameters.json, or prod__GlobalParameters.json depending on the environment release I need.

I'm not sure if this is the appropriate design as the document mentions (older mechanism), whereas my older mechanism of passing global parameters were via the ARMTemplateParametersForFactory.json file if I included this 

"Microsoft.DataFactory/factories/pipelines": {
        "properties": {
            "globalParameters": {
                "*": {
                    "value": "="
                }
            },
            "globalConfigurations": {
                "*": "="
            },

under arm-template-parameters-definition.json

Under ARMTemplateParametersForFactory.json, the parameters would appear like dataFactory_properties_globalParameters_

This all worked when setting "includeFactoryTemplate":true under publish_config.json (which is the deprecated option)

If the first option is in fact the new way for hydrating global parameters for a given environment, I'd prefer rewording this section Deploying using PowerShell (older mechanism) to something like Deploying using PowerShell (for updating global parameters with a given set of parameters) or something along those lines

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants