Suggestion: config/flag that restricts dorothy from installing further packages/dependencies.

[octavian-one]

Add system-wide configuration for automatic package installation behavior

Issue Type

Enhancement Request

Current Behavior

When running certain Dorothy commands (like get-filesystem), the system automatically attempts to install required packages without explicit user confirmation. In the observed case, the ripgrep utility installation process began automatically as part of the command execution.

Desired Behavior

Add a system-wide configuration option that allows users to control package installation behavior:

• Allow users to set whether package installations should be automatic or require confirmation
• Potentially support different levels of control:
  • Always ask
  • Never ask (automatic)
  • Ask only for new packages
  • Ask based on package size/impact

Use Case

Users may want to:

• Prevent unexpected package installations
• Review package requirements before installation
• Maintain stricter control over system modifications
• Understand dependencies being added to their system

Suggested Implementation

1. Add a new configuration option to support the different levels of control.
2. Modify existing commands to respect configuration
3. If set to "ask", present a clear prompt:

The command 'get-filesystem' requires the following packages:
- ripgrep from source: https://github.com/BurntSushi/ripgrep

Do you want to install these packages? [y/N]

Technical Notes

• Should maintain backward compatibility
• Could potentially be implemented per-package

Additional Context

This was observed while running get-filesystem, which triggered the setup_util_ripgrep installation process without user confirmation. While the automatic installation is convenient, it may not be desired in all environments or situations.

Let me know if you'd like me to clarify or expand on any part of this issue.

[octavian-one]

@molleweide what are your thoughts on this?

[balupton]

It is something possible, but I want to push back a little bit beforehand.

Ultimately, it does depend on your goal right? If your goal in this case is to get-filesystem, and ripgrep is needed for that, then would you give up on the goal by not installing ripgrep? It's akin to when we want to install package X via apt or homebrew etc, and it then says, to install package X we need to install these 20 other packages - how many people then select no to that prompt?

For implementation, we could have setup-util read a config var from setup.bash. The prompt should take into account whether the --optional flag was used in the setup-util invocation. The problem, and terrible UX will be when such a call is a few commands nested away, and not installing a package causes the callstack of several commands to fail - however, Dorothy is pretty good with error handling, so this shouldn't be an issue.

[molleweide]

At first glance i have to agree with Ben. If there is an easy way to signal to a command pipeline that it should not attempt to do something that could install packages going down the line. But I dont know how complex that would be to add.

It is either easy or complicated to add this. If it is just a simple variable then sure, but if it turns out to add lots of conditional complexity to pretty much all larger composed commands, then it feels like a lot of complexity for very little gain from a maintenance perspective.

But maybe it can be elegantly automated, however, I dont know if it is possible, and it is not urgent, but could however be a nice feature in the end..

[balupton]

An in-between alternative that's harmless could be listing dependencies inside the help text of commands, along with whether their installation is automated, and/or required/optional. Problem with this is it will have no visibility of nested deps, unless the help author manually goes hunting for such. Some tooling could probably be made to automate this to say 3 levels deep.

[molleweide]

that could prolly be automated by CI as well right?

[balupton]

that could prolly be automated by CI as well right?

Read my mind.

[molleweide]

it could also be a simple this command has deps or not to make it easy. I mean if a user takes issues with some dep X then it is there
own problem to investigate, but it would be easy to just make a global mode for dorothy that is restricted so that no deps are installed.

1. Flag commands if they will try to install deps yes or no.
2. User can specify certain packages that explicitly are not allowed to be installed maybe.

after these two then it beco{s just a whole ordeal in itself with different levels of granularity and then im like, just read the source instead..

[octavian-one]

@molleweide @balupton thanks for your input.

As a user, if I didn't know Ben I would have uninstalled Dorothy at that point and probably forgotten about it after that. The reason for this is that I am not used to anything acting as 'sudo apt-get' as a side effect. I may be mistaken, but I imagine those in an education environment, corporate environment, or any other place where there's system administration would feel the same.

After thinking on this more, as a user, I would also be fine if there was no per-package prompt and it was just an installation at Dorothy install time (ideally with a list of what will be installed). During updates, the screen could show the expected updates, removals, and installs.

You guys definitely know best as to what the priority of something like this should be. Thanks again!

[balupton]

There's def cross-over with #233

Is the issue the sudo usage? If ripgrep was installed to XDG bin home automatically without sudo, would that be an uninstall if Ben unknown? For solving #233 there will be a Dorothy install time config prompt for:

How would you like to handle commands that need sudo?

1. Prompt for the first invocation, then allow for the default duration
2. Prompt for every invocation
3. No sudo access at all, deny all invovations

While we could automate something to scan the deps of every command, to a certain nesting level, then show it in help text, and even say this dep is used by these commands; I'm thinking that then gets more complicated than a prompt configure; and would be too boring for me to code (PR/funding necessary). The immediate deps in immediate help tho would be easiest; then the config prompt at Dorothy install for whether all setup-util invocations require confirmation (there could also be a whitelist/blacklist of packages).

Tomorrow I'll write some proposals. Please continue contemplation.

[molleweide]

It is definitley possible to think more about this.

Dorothy is a dotfiels ecosystem as much as it is a shell/cli/bash dev lib.

And as a dotfiles system and lib that tries to solve problems it will subsequently also install necessary dependencies if required.

This might make some commands unusable if you cant/wont install any further deps.

From that perspective it might make sense to have some kind of flag that strictly forbids dorothy to install further programs.

This should be worked on over time.

1. add note to readme that informs user that dorothy is designed to try to install further deps.
2. if user disagrees, they will run dorothy in pure bash mode / restricted mode.
3. add more granularity over time.

[molleweide]

How would you like to handle commands that need sudo?

From a legal/corp perspective the solution could be to just state that dorothy will try to do these things..

and then just take it from there...

So instead of us first implementing some advanced analyzer, just add super explicit prompts that tell the user what to expect.

So that we manage expectations and make things more complicated later if necessary.

Cuz, if a newbie starts using dorothy, most things will be new to them and there is no way of handling all that. We can only tell them.
This is whats happening. Read up on this. otherwise....

[molleweide]

This is not a disagreement but just some additional philosphy: one goal of dorothy for me is also to setup a foundation for solving all possible problems you might have and so it is kinda inherent to the philosophy that dorothy would install necessary packages if required. So dorothy is like the interface and then you can do whatever inside of it and this means naturally that dorothy has and will install dependencies.

[octavian-one]

@balupton I can see the cross over to 233 (which I have read when it was discussed in the co-working session a while back).

If the implementation of identifying dependencies requires some form of analysis of the source to identify what may be installed, I'm happy to give the development of that a try.

I think there's a level of complexity in this 'package management' which may ultimately not fall within the scope of Dorothy, to @molleweide its ultimate goal is to accomplish what you need of it.

However, I want to share some other things I found in case it is of value:

There's an apt-file / auto-apt that may be of interested:

https://www.linuxquestions.org/questions/programming-9/bash-function-to-check-if-script-dependencies-are-installed-4175572568/#post5777118

Upon further research, I've discovered apt-cache search which looks promising: https://askubuntu.com/questions/160897/how-do-i-search-for-available-packages-from-the-command-line

However, if this dependency identification cannot be cleanly implemented then I propose that a 'proxy' for it is not developed but that the Github / program simply has a list of dependencies that is documented.

[balupton]

@octavian-one one other aspect here, is that Dorothy, or any open-source project, could rm -Rf /. Whether or not they do or not is based on the trust and reputation of those who do open-source. Is there a difference between running Dorothy code, and say running ripgrep code?

Perhaps security isn't the concern here; I haven't seen that shine through in this report, or the sudo report, or any other issue. What is shining through in this report is "Dorothy did something I didn't expect" in which case we need to manage expectations. For the sudo issue, what shone through is "sudo prompts on a non-root system is extremely annoying" in which we need to just skip sudo.

[balupton]

There's an apt-file / auto-apt that may be of interested:

These would be doubling up on Dorothy's setup-util technology. For instance, dependency installs happen via say setup-util-ripgrep --quiet --optional, this command then calls setup-util with the appropriate arguments for each package system that ripgrep supports, allowing a portable and robust package system. It really is amazing.

The automation @molleweide and I were discussing would be something that ensures these setup-util-* calls within the command source, are reflected within the help text of the command, and vice versa - triggering CI failures if say setup-util-ripgrep was called but not mentioned in the help text, or if a dependency was mentioned in the help text but not actually used in the source code.

Such detection could be extended however, to allow for visibility with the confirm any setup-util invocation. For instance, you may not want to install ripgrep just for get-filesystem, but if you also were aware that ripgrep is also used by:

• config-edit
• dorothy (for tests)
• get-filesystem

Then you may be interested in it. Such could be done by syncing source code analysis with the help text, but also say a $DOROTHY/dependencies.json file or the like that has say:

{
    "dependents":  {
        "config-edit": {
            "ripgrep": {
                "optional": false
            }
        },
        "dorothy": {
            "ripgrep": {
                "optional": false
            }
        },
        "get-filesystem": {
            "ripgrep": {
                "optional": false
            }
        }
    },
    "dependencies": {
        "ripgrep": {
            "config-edit": {
                "optional": false
            },
            "dorothy": {
                "optional": false
            },
            "get-filesystem": {
                "optional": false
            }
        }
    }
}

Which can then be sourced by setup-util to offer additional information in the confirm prompt.

ripgrep is actually a bad example here, as it has largely been replaced with Dorothy's own echo-regexp, which has deno as a dependency. As soon as echo-regexp supports --fixed-strings then ripgrep can be dropped from config-edit and get-filesystem and will only be used in the dorothy for its check/lint/test commands (from memory). Another reason why ripgrep is a bad example, is it actually calls setup-util-ripgrep via a ripgrep.bash sourcing.

However, this also illustrates the complexity of such. As echo-regexp depends on Deno, so to raise awareness of this deno dependency for all commands that also call echo-regexp which is 50 files (1 level of nesting) that is an ordeal, let alone when we get to multiple level of nestings. Which at which point, I'm not sure the complexity is justified compared to alternatives.

[balupton]

So here are the proposals:

setup-util config option for confirmations

Introduce a SETUP_UTIL_CONFIRMATION='always' # default, unknown config option to setup.bash.

When always any installation or uninstallation action by setup-util will trigger a confirmation prompt.

When default confirmation will only occur if the invoker of setup-util requested it.

When unknown confirmation will only occur if the invoker of setup-util requested it, or if the utility is not included within the SETUP_UTILS array in setup-bash.

Cost considerations:

• setup-util currently does not read any configuration, this changes that. Impact should be minimal.
• This opens the door to making setup-util heavier, say by:
  • inlining get-installer functionality (caveat here is get-installer requires bash v4 - however its associated array usage could be a JSON file)
  • introducing a --save flag that adds the utility to SETUP_UTILS if installed, or removes it if uninstalled.

Requirements checklist:

• raises awareness for all dependencies, included nested dependencies
• minimal overhead

raise awareness of dependencies in help text

If a command has a setup-util-* call, then it should list this in a DEPENDENCIES section in its help text, along with whether it is optional, or required.

Cost considerations:

• help text and code can get out of sync, CI could be written to keep them in sync
• doesn't help users who don't read the help text first
• doesn't solve case for nested dependencies, without significant maintainer burden or significant CI complexity: Suggestion: config/flag that restricts dorothy from installing further packages/dependencies. #263 (comment)

Requirements checklist:

• fails to raise awareness of nested dependencies without significant back-and-forth maintainer overhead
• notable maintainer overhead if manual
• notable CI overhead if automatic

update readme.md to reflect dependency install capabilities

Currently Dorothy has this in its README header:

automatic installation and updating of your specified packages

Which is inaccurate, as it doesn't mention that packages that are also dependencies of a command you wanted to run will also be automatically installed/updated. This can cause a violation of expectation, and consequently of trust.

As it stands right now, this also needs to be added:

automation management of command dependencies

Which wording and be updated based on whether the setup-util config option is added.

[octavian-one]

@balupton regarding a nefarious command such as delete everything... you raise a good point. Running un-trusted applications could be sandboxed (to a certain degree) with AppArmour. An organization should already be preventing rm -rf, apt-get, etc, and if they're not then it's up to their users and policies to prevent un-authorized code execution.

A quick aside: I had previously missed the meaning of 'would that be an uninstall if Ben unknown?' which is pretty funny. It sounds like a Dorothy command! It also seems I didn't refresh GitHub before my comment this morning (leading me to miss out on @molleweide 's additional notes).

I wasn't aware of the complexity of setup-util and its full capabilities, which I must agree are amazing.

The proposal you made is robust and from the background of what drove me to create this issue, I want to note that the readme update itself would alleviate the trust concern I raised.

You can imagine (as a somewhat naive user) what was going through my mind when I ran get-filesystem only to see some flashing lines of apt-get which then disappeared. It is this disappearing act which greatly contributed to my unease (leaving me thinking 'did I get my filesystem or did someone else?').

Thank you both for addressing and collaborating on the design of the proposed enhancement.

[balupton]

One thing that I've done locally is add a dorothy commands action to the dorothy command (note that this calls setup-util-eza --quiet --optional 😉 )

I could update it so that dorothy commands get-filesystem dumps the help text and source code, and perhaps even a GitHub Copilot summary:

---

What does this command do?

The get-filesystem command is a shell script designed to determine the filesystem type of a given mount source, target, device, or path. Here's a breakdown of the visible part of the script:

Breakdown

1. Options and Arguments:

   • --source=<source>: Specifies a mount source.
   • --target=<target>: Specifies a mount target.
   • --device=<device>: Specifies a device.
   • <path>: Attempts automatic detection of the appropriate type.

2. Error Handling:

   • If the script is called with incorrect arguments, it prints an error message and returns an error code 22 (Invalid argument).

3. Dependencies:

   • The script sources another script located at $DOROTHY/sources/ripgrep.bash, which likely contains helper functions or configurations.

4. Helper Functions:

   • get_filesystem_from_source: Determines the filesystem type from a mount source.
     • Uses the mount command and rg (ripgrep) to parse the output.
     • Different parsing logic for macOS (is-mac) and other systems.
   • get_filesystem_from_target: Determines the filesystem type from a mount target.
     • Similar logic to get_filesystem_from_source, but for mount targets.

Example Usage

Determine Filesystem from Source

get-filesystem --source=/dev/sda1

Determine Filesystem from Target

get-filesystem --target=/mnt/data

Explanation of Helper Functions

get_filesystem_from_source

• macOS:
  • Uses mount to list mounted filesystems.
  • Filters lines containing the source ($1) using rg.
  • Extracts the filesystem type using a regex pattern.
• Other Systems:
  • Similar logic but uses a different regex pattern to extract the filesystem type.

get_filesystem_from_target

• macOS:
  • Uses mount to list mounted filesystems.
  • Filters lines containing the target ($1) using rg.
  • Extracts the filesystem type using a regex pattern.
• Other Systems:
  • Similar logic but uses a different regex pattern to extract the filesystem type.

Conclusion

The get-filesystem script is a utility to determine the filesystem type of a specified mount source, target, device, or path. It uses the mount command and rg (ripgrep) for parsing the output and supports different parsing logic for macOS and other systems.

[octavian-one]

@balupton the list with the potential to get the help text of commands is phenomenal. This is a great shell experience and is what I'm used to in other packages. The summary could be beneficial, but I think that may add more overhead at this time in reviewing the output and keeping it up to date. One example of it being a little vague was 'The script sources another script located at $DOROTHY/sources/ripgrep.bash, which likely contains helper functions or configurations.'

In the future, such summaries could even be available in a documentation wiki (Github, etc), but I just think it would add a huge time commitment to our valuable contributors (ahem mostly you) which may best addressed down the road when adoption is greater (or the outcry for such docs rises).

[molleweide]

Should we maybe just start a wiki now and put some placeholder sections like "Getting started" and "FAQ", and then we can populate as time goes on.

And link to them from readme

and also link to the roadmap from the readme or wiki

[balupton]

@molleweide nah no wiki, they are a pain to keep updated, as they can get out of sync way too easily with the codebase. Keep everything in the readme or the docs folder, then everything that is out of date easily shows up when I search VSCode for things.

Originally Dorothy did have a wiki, and GitHub Discussions, but they were archived for this reason - everything got out of date because there was no visibility next to the code to keep them updated.

[molleweide]

ah ok make sense