Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create droppath option in ssm_parameter.py #1756

Merged

Conversation

jacobcrell
Copy link
Contributor

@jacobcrell jacobcrell commented Sep 14, 2023

SUMMARY

In the event that there exists two parameters in Parameter Store path/to/params/foo/bar/param and path/to/params/x/y/param, a user attempting to recursively pull all parameters under the path/to/params path with the command {{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', shortnames=true, bypath=true, recursive=true ) }}" will be unable to use the shortnames argument without creating a conflict between the two param results. Only one param key will exist in the resulting dictionary.

This is problematic for users that wish to lookup a parameter hierarchy with repetitive names in the final value of the parameter path. In this scenario, users are currently forced to use the fully qualified parameter name when interacting with their dictionary results.

This pull request introduces the droppath option, which will drop the common lookup path from the names of all parameters. In the scenario above, the resulting dictionary would contain both a foo/bar/param and a /x/y/param allowing users to refer to them by their minimally differentiated name.

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME
  • lookup
ADDITIONAL INFORMATION
Step by step

Setup

  • Create parameter path/to/params/foo/bar/param with value abc
  • Create parameter path/to/params/x/y/param with value 123

Run shortnames

    • name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
      debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', shortnames=true, bypath=true, recursive=true ) }}"
  • Observe result {'param': '123'}

Run droppath

    • name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
      debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', droppath=true, bypath=true, recursive=true ) }}"
  • Observe result {'/foo/bar/param': 'abc', '/x/y/param': '123'}

Run both

    • name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
      debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', droppath=true, shortnames = true, bypath=true, recursive=true ) }}"
  • Observe error shortnames and droppath are mutually exclusive. They cannot both be set to true.

@github-actions
Copy link

github-actions bot commented Sep 14, 2023

Docs Build 📝

Thank you for contribution!✨

This PR has been merged and your docs changes will be incorporated when they are next published.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/123852a00e6b438c83999d63178ec865

ansible-galaxy-importer FAILURE in 3m 56s
✔️ build-ansible-collection SUCCESS in 12m 41s
✔️ ansible-test-splitter SUCCESS in 4m 35s
integration-amazon.aws-1 FAILURE in 6m 34s
Skipped 43 jobs

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/7b6889acd3f14865956cfede636d7e0d

✔️ ansible-galaxy-importer SUCCESS in 8m 48s
✔️ build-ansible-collection SUCCESS in 14m 16s
✔️ ansible-test-splitter SUCCESS in 5m 10s
integration-amazon.aws-1 FAILURE in 8m 30s
Skipped 43 jobs

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/98d7b132f5564d1fac455e2e7d42498b

✔️ ansible-galaxy-importer SUCCESS in 3m 38s
✔️ build-ansible-collection SUCCESS in 13m 18s
✔️ ansible-test-splitter SUCCESS in 4m 51s
✔️ integration-amazon.aws-1 SUCCESS in 5m 56s
Skipped 43 jobs

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/13f13b48384442d0a49a11236bcfdd97

✔️ ansible-galaxy-importer SUCCESS in 4m 21s
✔️ build-ansible-collection SUCCESS in 12m 48s
✔️ ansible-test-splitter SUCCESS in 4m 41s
integration-amazon.aws-1 FAILURE in 7m 21s
Skipped 43 jobs

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/55bc674f212a483f949ea467e458d56c

✔️ ansible-galaxy-importer SUCCESS in 5m 07s
✔️ build-ansible-collection SUCCESS in 13m 09s
✔️ ansible-test-splitter SUCCESS in 5m 15s
integration-amazon.aws-1 FAILURE in 7m 43s
Skipped 43 jobs

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/d202f198e3014fbaaf5e17f2dab28123

✔️ ansible-galaxy-importer SUCCESS in 5m 04s
✔️ build-ansible-collection SUCCESS in 12m 57s
✔️ ansible-test-splitter SUCCESS in 4m 43s
integration-amazon.aws-1 FAILURE in 7m 33s
Skipped 43 jobs

Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/6f89e2f855e44de59b4a11e3fa299062

✔️ ansible-galaxy-importer SUCCESS in 5m 28s
✔️ build-ansible-collection SUCCESS in 14m 13s
✔️ ansible-test-splitter SUCCESS in 6m 18s
integration-amazon.aws-1 FAILURE in 5m 27s
Skipped 43 jobs

Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/b493b18f97034cc186fd8a66b1f01113

✔️ ansible-galaxy-importer SUCCESS in 5m 15s
✔️ build-ansible-collection SUCCESS in 15m 32s
✔️ ansible-test-splitter SUCCESS in 6m 29s
integration-amazon.aws-1 FAILURE in 10m 10s
Skipped 43 jobs

@tremble
Copy link
Contributor

tremble commented Aug 30, 2024

Sorry it's taken so long to review this. I've rebased and fixed up the integration tests.

Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/6d4bfc2154504f1793a9ef1980e93f37

✔️ ansible-galaxy-importer SUCCESS in 5m 26s
✔️ build-ansible-collection SUCCESS in 10m 39s
✔️ ansible-test-splitter SUCCESS in 4m 17s
✔️ integration-amazon.aws-1 SUCCESS in 6m 04s
Skipped 43 jobs

@tremble tremble added mergeit Merge the PR (SoftwareFactory) backport-8 PR should be backported to the stable-8 branch labels Aug 30, 2024
Copy link
Contributor

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/06bccf2b33a8476a8f7ebc767ef5ff43

✔️ ansible-galaxy-importer SUCCESS in 3m 42s
✔️ build-ansible-collection SUCCESS in 10m 34s
✔️ ansible-test-splitter SUCCESS in 4m 18s
✔️ integration-amazon.aws-1 SUCCESS in 4m 52s
Skipped 43 jobs

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit c535e63 into ansible-collections:main Aug 30, 2024
37 checks passed
Copy link

patchback bot commented Aug 30, 2024

Backport to stable-8: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-8/c535e630811d9e8aaa6aad639cf0c804b9c02eb8/pr-1756

Backported as #2274

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback bot pushed a commit that referenced this pull request Aug 30, 2024
SUMMARY
In the event that there exists two parameters in Parameter Store path/to/params/foo/bar/param and path/to/params/x/y/param, a user attempting to recursively pull all parameters under the path/to/params path with the command {{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', shortnames=true, bypath=true, recursive=true ) }}" will be unable to use the shortnames argument without creating a conflict between the two param results.  Only one param key will exist in the resulting dictionary.
This is problematic for users that wish to lookup a parameter hierarchy with repetitive names in the final value of the parameter path. In this scenario, users are currently forced to use the fully qualified parameter name when interacting with their dictionary results.
This pull request introduces the droppath option,  which will drop the common lookup path from the names of all parameters. In the scenario above, the resulting dictionary would contain both a foo/bar/param and a /x/y/param  allowing users to refer to them by their minimally differentiated name.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME

lookup

ADDITIONAL INFORMATION
Step by step
Setup

Create parameter path/to/params/foo/bar/param with value abc
Create parameter path/to/params/x/y/param with value 123

Run shortnames

name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', shortnames=true, bypath=true, recursive=true ) }}"

Observe result  {'param': '123'}

Run droppath

name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', droppath=true, bypath=true, recursive=true ) }}"

Observe result  {'/foo/bar/param': 'abc', '/x/y/param': '123'}

Run both

name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', droppath=true, shortnames = true, bypath=true, recursive=true ) }}"

Observe error shortnames and droppath are mutually exclusive. They cannot both be set to true.

Reviewed-by: Mark Chappell
(cherry picked from commit c535e63)
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Aug 30, 2024
This is a backport of PR #1756 as merged into main (c535e63).
SUMMARY
In the event that there exists two parameters in Parameter Store path/to/params/foo/bar/param and path/to/params/x/y/param, a user attempting to recursively pull all parameters under the path/to/params path with the command {{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', shortnames=true, bypath=true, recursive=true ) }}" will be unable to use the shortnames argument without creating a conflict between the two param results.  Only one param key will exist in the resulting dictionary.
This is problematic for users that wish to lookup a parameter hierarchy with repetitive names in the final value of the parameter path. In this scenario, users are currently forced to use the fully qualified parameter name when interacting with their dictionary results.
This pull request introduces the droppath option,  which will drop the common lookup path from the names of all parameters. In the scenario above, the resulting dictionary would contain both a foo/bar/param and a /x/y/param  allowing users to refer to them by their minimally differentiated name.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME

lookup

ADDITIONAL INFORMATION
Step by step
Setup

Create parameter path/to/params/foo/bar/param with value abc
Create parameter path/to/params/x/y/param with value 123

Run shortnames



name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', shortnames=true, bypath=true, recursive=true ) }}"


Observe result  {'param': '123'}

Run droppath



name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', droppath=true, bypath=true, recursive=true ) }}"


Observe result  {'/foo/bar/param': 'abc', '/x/y/param': '123'}

Run both



name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
debug: msg="{{ lookup('amazon.aws.aws_ssm', '/PATH/to/params', region='ap-southeast-2', droppath=true, shortnames = true, bypath=true, recursive=true ) }}"


Observe error shortnames and droppath are mutually exclusive. They cannot both be set to true.

Reviewed-by: Mark Chappell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8 PR should be backported to the stable-8 branch mergeit Merge the PR (SoftwareFactory)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants