The interplay between host_vars and group_vars dirs and inventory dirs is not fully documented

[kpinc]

Hi,

This is more-or-less a review of PR #2416, with an eye towards what PR #2386 was intended to improve. Apologies for doing this after-the-fact.

In no particular order:

Nothing explicitly says that when you specify a directory in which to find inventory, that inventory is collected in a recursive breath-first manner from all sub-directories. Although this is implied by "Files will be read and loaded in alphabetically sorted order from the top directory down." , the reader has to pay attention, and do a certain amount of assuming, to realize that that means that sub-directories, and their sub-sub-directories, etc., are searched for inventory files. Something more explicit would be better.

Nothing explicitly says that host_vars and group_vars dirs are searched for only in the directory given to ansible as the top-level dir in which to search for inventory. This is almost implied by an example that starts with "If your inventory file at /etc/ansible/hosts ...". But not really because I can invoke ansible with
ansible-playbook -i /etc ... and while the example will still use the same inventory files, the group_vars and host_vars dirs will not be used. (And yes, looking through all of /etc would be bad, but that's not the point.) The example implies that it matters where the inventory file is found, but this is not the case. In the end, it is left undocumented that, although inventory files are searched for in sub-directories, this is not the case for the host_vars and group_vars directories.

I am not a fan of "documentation by example". I believe that the text should be explicit about the behavior documented. Rather than just an example it would be best to have sentences that document behavior. So the example starting with "If your inventory file at /etc/ansible/hosts ..." should be supplemented with sentences that document behavior. The example can then clarify and illustrate the behavior. The reader should not have to guess at the behavior the example is supposed to illustrate.

[oraNod]

@bcoca I've assigned this to you since it follows up on one of your PRs. Hope that's OK. Could you please take a look and provide your thoughts? Thanks.

[bcoca]

so

The host_group_vars vars plugin loads host and group variable files by searching paths relative to the inventory source or the playbook file. If your inventory file at /etc/ansible/hosts contains a host named ‘foosball’ that belongs to two groups, ‘raleigh’ and ‘webservers’, that host will use variables in YAML files at the following locations:

to

The host_group_vars vars plugin loads host and group variable files by searching paths adjacent to the inventory source or the playbook file. If your inventory file at /etc/ansible/hosts contains a host named ‘foosball’ that belongs to two groups, ‘raleigh’ and ‘webservers’, that host will use variables in YAML files at the following locations:

?

[kpinc]

On Tue, 25 Mar 2025 15:09:57 -0700 Brian Coca ***@***.***> wrote: bcoca left a comment (ansible/ansible-documentation#2462) so ``` The host_group_vars vars plugin loads host and group variable files by searching paths ... ``` to

"repeated" I don't see a difference between the 2 bits of text quoted. Thanks for responding. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein

[bcoca]

I changed 'relative' to 'adjacent' which I believe clarifies that the source must be at the same level

[kpinc]

On Wed, 26 Mar 2025 08:07:37 -0700 Brian Coca ***@***.***> wrote: bcoca left a comment (ansible/ansible-documentation#2462) I changed 'relative' to 'adjacent' which I believe clarifies that the source must be at the same level

Sorry for missing that. But the source does not have to be at the same level. At least not at the same level as the inventory file(s). It has to be a file contained within the directory in which ansible starts its recursive search for inventory files -- that'd be the -i value in, say, an ansible-playbook invocation. (Or similar in the ansible config file, etc.) So anything said about the inventory files themselves is not really relevant. "The directory in which ansible starts its recursive search for inventory files", whatever you call that, must be the subject of a sentence if this behavior is to be clear. Or that's my assessment anyway. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein

[bcoca]

why the documentation makes a distinction between 'inventory sources' vs 'inventory files', the first is what you supply to ansible as a 'source' and becomes the basis for search for vars plugins, the 2nd has actual 'content' for the plugins to consume.

[kpinc]

On Wed, 26 Mar 2025 11:48:05 -0700 Brian Coca ***@***.***> wrote: bcoca left a comment (ansible/ansible-documentation#2462) why the documentation makes a distinction between 'inventory sources' vs 'inventory files', the first is what you supply to ansible as a 'source' and becomes the basis for search for vars plugins, the 2nd has actual 'content' for the plugins to consume.

I see. I found https://docs.ansible.com/ansible/latest/dev_guide/developing_inventory.html#inventory-sources, which defines "inventory source" to be: "Inventory sources are the input strings that inventory plugins work with. An inventory source can be a path to a file or to a script, or it can be raw data that the plugin can interpret." (It wouldn't hurt, by the by, to link to the definition of "inventory source", where the term is used. Pretty much everywhere, IMO. But, although I think it would help, this is not really relevant.) (Also offtopic, or at least not mentioned in the issue, I now have trouble with the sentence at https://docs.ansible.com/ansible/latest/inventory_guide/intro_inventory.html#organizing-inventory-in-a-directory: "You can consolidate multiple inventory sources in a single directory." Replacing "sources" with "files" makes sense, but may not be accurate or convey the intended meaning. (?) Anyway, since a source is a string input to ansible it does not make real sense (to me) to talk about _sources_ in the filesystem being "consolidated". Don't sources have to be strings that come from the command line or configuration files, not filesystem objects? Because otherwise they'd be inventory files instead of inventory sources? Please don't bother replying if you don't care to.) Examining the sentence: "The host_group_vars vars plugin loads host and group variable files by searching paths relative to the inventory source or the playbook file." Changing "relative" to "adjacent" is not going to help. The example does not say if the inventory source is a file or a directory. If the inventory source is a path to a file, it is true that host_vars and group_vars dirs must be adjacent to that file. But what if the inventory source is a directory, say, /etc/ansible in this case? The host_vars and group_vars directories must be immediately inside the inventory source directory, not adjacent to it, to be used. In this case the sentence is wrong. So your example does not fully explain how the host_vars and group_vars directories must be placed. You need a 2nd example. Or, I believe, explicit statements that describe how things work with enough examples to cover non-obvious use-cases. Because inventory sources are not always directories, a single sentence is not going to be enough to explain how things work. How about this? Instead of the paragraph "The host_group_vars vars plugin loads host and group variable files by searching paths relative to the inventory source or the playbook file. If your inventory file at ``/etc/ansible/hosts`` contains a host named 'foosball' that belongs to two groups, 'raleigh' and 'webservers', that host will use variables in YAML files at the following locations:" Have 2 paragraphs (RST Sphinx markup here): The host_group_vars plugin loads host and group variable files by looking for ``host_vars`` and ``group_vars`` directories in three places. First, in the directory containing the playbook file. Second, when the :ref:`inventory source <inventory_sources>` is a file, in the directory containing the inventory source. And third, when the inventory source is a directory, in the inventory source directory, but, unlike inventory file discovery, not within any sub-directories. For example, if your inventory source is either the file at ``/etc/ansible/hosts`` or the directory ``/etc/ansible/``, and a host named 'foosball' is given, that belongs to two groups, 'raleigh' and 'webservers', that host will use variables in YAML files at the following locations: Note that the above still needs further explanation, since it does not say what happens if there is a host_vars or group_vars directory both in the same directory as the playbook _and_ either in the same directory as a inventory source that is an inventory file or in an inventory source directory. It also does not say what happens if there are multiple inventory sources that would cause multiple host_vars/group_vars dirs to be found, although perhaps that's explained elsewhere. (Specifically, are they all used and if so, in what order?) If so, a link is called for. Finally, my suggested text implies an ordering that I made up without knowing if it is correct. (One way or another, order matters.) (Note also, I've hyperlinked the first use of "inventory source" above, but it probably makes sense to hyperlink the term in the first sentence of the section instead of where I did it here. Unless it's appropriate to hyperlink every occurrence?) To be through, add another paragraph after the example that says: However, if the inventory source of the above example was instead ``/etc/``, the inventory file at ``/etc/ansible/hosts`` would be found and used, but neither the ``host_vars`` or the ``group_vars`` directories would be used. With this additional paragraph, all the corner cases I complain about in the issue are both stated and explained. As long as I'm suggesting actual text, to address the other unspecified behavior raised in the issue, how about instead of: "Files will be read and loaded in alphabetically sorted order from the top directory down." Use: "Files are read and loaded in an alphabetically sorted order in a recursive manner. All files and directory names are sorted alphabetically using the collating algorithm and character encoding defined by the target system's locale. Files are then read and loaded as they are encountered in this ordered list of names, with directories immediately processed recursively when a directory name is seen in the sorted list. So the files contained within a sub-directory, or its sub-directories, are read and loaded before any files remaining in the list." (Note also the use of present tense.) Unless of course files are processed _first_, and then recursion into alphabetically sorted directories occurs. Which is either depth or breadth-first. Or the locale is that of the ansible controller. Or whatever actually happens. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein