Allow to override nested array / maps properties

[sschuberth]

          > I guess we could support `databases[1]__port = 9301` ?

I'd still be interested to override single properties within a list of a config read from a file via CLI options like that, without the need to declare "placeholder properties" that refer to environment variables. Is something like that supported by now?

Originally posted by @sschuberth in #144 (comment)

[sschuberth]

More specifically, it would be great if MapPropertySource and CommandLinePropertySource would support the "key" to contain an index like [1] to be able to override single values nested within an array list. (Even is that value was not present originally, so it's more "add" than "override"; in that aspect this issue is different from #144, where you can work with environment placeholders.)

[sschuberth]

@sksamuel I'm willing to work on this, but I have no clue where to start. Is my approach outlined above even feasible to implement?

[sksamuel]

I think your approach for map property source and cli property source are perfectly reasonable.
We already do something some what similar in the aws preprocessor with [] used for json fields.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[sschuberth]

While I didn't have time to work on this yet, I believe it still makes sense as a feature to add.

[sschuberth]

aws preprocessor with [] used for json fields.

Maybe by now https://github.com/eygraber/JsonPathKt could be leveraged...

[sksamuel]

Not sure we even need that. Just need to take [n] and apply that to the list ?

[sschuberth]

Not sure we even need that. Just need to take [n] and apply that to the list ?

For lists you're probably right. But we basically have the same issue also for overriding values in a map. And there, in order to refer to the key's name in the map, maybe JsonPathKt would be helpful.

[rocketraman]

For reference, Spring Boot does environment variable binding to object lists by the _n_ syntax. See https://docs.spring.io/spring-boot/reference/features/external-config.html#features.external-config.typesafe-configuration-properties.relaxed-binding.environment-variables.

It'd probably be a good idea to implement the same in Hoplite for compatibility / consistency.

Also, from https://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap08.html:

Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation; applications shall tolerate the presence of such names.

So while characters like [ and ] are allowed, they would be unusual and may cause issues in some systems. Sticking with uppercase letters, digits, and _ (as Spring Boot does) would be best.

It seems like JsonPathKt would be overkill / too complicated syntax for this use case.

I'm not sure what you were thinking about for maps. I don't think Spring Boot supports that. That's not to say that Hoplite couldn't. Maybe the syntax for a map key could be double-underscore or something.

[sschuberth]

I'm not sure what you were thinking about for maps.

I didn't have a concrete syntax in mind yet. I only know our requirement to be able to set Map<String, String> configuration items from the CLI 😄