diff --git a/blog/2024-04-30-nushell_0_93_0.md b/blog/2024-04-30-nushell_0_93_0.md index 516d56c4d21..fccddf11d76 100644 --- a/blog/2024-04-30-nushell_0_93_0.md +++ b/blog/2024-04-30-nushell_0_93_0.md @@ -5,6 +5,7 @@ author_site: https://twitter.com/nu_shell author_image: https://www.nushell.sh/blog/images/nu_logo.png excerpt: Today, we're releasing version 0.93.0 of Nu. This release adds... --- + # Nushell 0.93.0 @@ -12,6 +13,7 @@ excerpt: Today, we're releasing version 0.93.0 of Nu. This release adds... Nushell, or Nu for short, is a new shell that takes a modern, structured approach to your command line. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful command line pipelines. + Today, we're releasing version 0.93.0 of Nu. This release adds... # Where to get it @@ -26,12 +28,22 @@ As part of this release, we also publish a set of optional plugins you can insta # Table of content - [*Themes of this release / New features*](#themes-of-this-release-new-features-toc) + - [*Redesigned plugin management commands*](#redesigned-plugin-management-commands-toc) + - [*New plugin cache format*](#new-plugin-cache-format-toc) + - [*Improved terminal interaction for plugins*](#improved-terminal-interaction-for-plugins-toc) - [*Hall of fame*](#hall-of-fame-toc) - [*Bug fixes*](#bug-fixes-toc) - [*Enhancing the documentation*](#enhancing-the-documentation-toc) - [*Stricter rules around setting environment variables*](#stricter-rules-around-setting-environment-variables-toc) - [*Our set of commands is evolving*](#our-set-of-commands-is-evolving-toc) - [*New commands*](#new-commands-toc) + - [*`plugin add`*](#plugin-add-toc) + - [*`plugin rm`*](#plugin-rm-toc) + - [*`plugin use`*](#plugin-use-toc) + - [*`from msgpack`*](#from-msgpack-toc) + - [*`to msgpack`*](#to-msgpack-toc) + - [*`from msgpackz`*](#from-msgpackz-toc) + - [*`to msgpackz`*](#to-msgpackz-toc) - [*Changes to existing commands*](#changes-to-existing-commands-toc) - [*`with-env`*](#with-env-toc) - [*`load-env`*](#load-env-toc) @@ -46,7 +58,11 @@ As part of this release, we also publish a set of optional plugins you can insta - [*`into-filesize`*](#into-filesize-toc) - [*`explain`*](#explain-toc) - [*Deprecated commands*](#deprecated-commands-toc) + - [*`register`*](#register-toc) - [*Removed commands*](#removed-commands-toc) + - [*For plugin developers*](#for-plugin-developers-toc) + - [*New protocol features*](#new-protocol-features-toc) + - [*New engine calls*](#new-engine-calls-toc) +## Redesigned plugin management commands [[toc](#table-of-content)] + +::: warning Deprecation warning +`register` is now deprecated and will be removed in the next major release. Read this section for alternatives. +::: + +We've changed the way plugins are managed in Nushell in this release. In 0.92 and earlier, plugins were added with `register`, e.g.: + +```nushell +> register ~/.cargo/bin/nu_plugin_gstat +``` + +This ran the plugin to get the signatures of the commands it supports, and then did two things: it added them to your `$nu.plugin-file` (previously called `plugin.nu`), so that they would be ready next time `nu` is run without having to ask the plugin again, and it also added them to your scope so you could use them immediately. While this was convenient, it had the drawback of always running the plugin at parse time (see issue [#11923](https://github.com/nushell/nushell/issues/11923)), which was less than ideal. + +We now have two commands to replace `register` that break this functionality into two steps: + +1. [`plugin add`](#plugin-add-toc), a normal command, which runs the plugin to get command signatures and adds them to your `$nu.plugin-file`, replacing any existing plugin definition with the same name +2. [`plugin use`](#plugin-use-toc), a parser command, which loads the plugin command signatures from your `$nu.plugin-file` and makes them available in scope. + +The previous example would now be equivalent to this, at the REPL: + +```nushell +# Add the plugin's commands to your plugin registry file: +> plugin add ~/.cargo/bin/nu_plugin_gstat + +# Load it into scope: +> plugin use gstat +``` + +Note that this _will not_ work in a script in this order, as `plugin use` is a [parser command](/book/thinking_in_nu.md#think-of-nushell-as-a-compiled-language) and would get evaluated first. If you would like to make sure a certain plugin is available in a script, there are a few options: + +1. Prepare a plugin registry file in advance, and use `--plugin-config` with `plugin use`: + ```nushell + plugin use --plugin-config /path/to/custom.msgpackz your_required_plugin + ``` +2. The same thing, but pass the `plugin config` to `nu`: + ```nushell + nu --plugin-config /path/to/custom.msgpackz your-script.nu + ``` +3. Pass a NUON list to `--plugins` (new in 0.93) to `nu`, to use plugins without a registry file: + + ```nushell + nu --plugins '[/path/to/nu_plugin_your_required_plugin]' your-script.nu + + # or you can use `to nuon` from inside nu: + let plugins = [ + '/path/to/nu_plugin_foo' + '/path/to/nu_plugin_bar' + ] + nu --plugins ($plugins | to nuon) your-script.nu + ``` + +The last option will run the plugin to get signatures at startup, without modifying the user's registry file, and can even be used when `--no-config-file` is specified. + +Finally, we have also added [`plugin rm`](#plugin-rm-toc), which removes a plugin from the registry file. This was something that previously needed to be done manually in a text editor, but is made possible by the new format. + +## New plugin registry format [[toc](#table-of-content)] + +::: warning Breaking change +See a full overview of the [breaking changes](#breaking-changes) +::: + +The plugin registry format has been changed from the previous Nushell script format (containing a sequence of `register` commands) to a custom, declarative format based on [MessagePack](https://msgpack.org/) compressed with [brotli](https://github.com/google/brotli). This new format is called `msgpackz`, and the default plugin file is `plugin.msgpackz` instead of `plugin.nu`. + +Your existing `plugin.nu` file will be automatically upgraded to `plugin.msgpackz` the first time version 0.93 runs. + +The declarative format brings a number of benefits: + +1. It can be edited more structurally, allowing us to replace the entire definition of one plugin at once, which is more logically consistent if a plugin removes commands from one version to the next, and also allowing us to support new commands like [`plugin rm`](#plugin-rm-toc). +2. It supports adding metadata about plugins as a whole rather than just the command signatures, if we need to. +3. It contains the Nushell version that generated it, allowing us to potentially fix issues between versions. +4. It fails gracefully - if something about a particular plugin can't be parsed because of a change to the serialization format, other plugins that don't have that issue can still load. + +We chose to use compressed MessagePack because it is very fast to load, and the file is loaded every time `nu` starts, so with a lot of plugins or plugins that expose a lot of commands (e.g. the new `polars` plugin), this was a big problem for startup times in general. + +The most significantly _breaking_ part of this breaking change is that the following is no longer possible: + +```nushell +source $nu.plugin-file +``` + +You can use the [`plugin use`](#plugin-use-toc) command instead to (re-)load a specific plugin from the plugin registry file: + +```nushell +> plugin use gstat +``` + +## Improved terminal interaction for plugins [[toc](#table-of-content)] + +Plugins are now able to run over a UNIX domain socket or Windows named pipe instead of `stdin`/`stdout` if they support it ([#12448](https://github.com/nushell/nushell/pull/12448)). All Rust plugins support this mode by default, but it is optional; plugins written in other languages are not forced to implement this, and the stdio mode is still always attempted. The [feature is advertised as `LocalSocket`](/contributor-book/plugin_protocol_reference.md#localsocket-feature) and the upgrade is automatic when supported by the plugin and confirmed to be functional. + +This frees up stdio for interaction with the user's terminal, which makes plugins that rely on that, including terminal UI plugins like @amtoine's [`nu_plugin_explore`](https://github.com/amtoine/nu_plugin_explore) simpler and more reliable to implement. + +We also added [engine calls](/contributor-book/plugin_protocol_reference.md#enterforeground-engine-call) for moving the plugin into the terminal controlling (foreground) process group, mainly for Unix platforms. This was also necessary to fully support terminal UI plugins, after the last release's new persistence functionality caused plugins to run in a new, background process group by default. + ## Hall of fame [[toc](#table-of-content)] + ### Bug fixes [[toc](#table-of-content)] + Thanks to all the contributors below for helping us solve issues and bugs :pray: -| author | description | url | +| author | description | url | | ------------------------------------ | ----------- | ------------------------------------------------------- | -| [@author](https://github.com/author) | ... | [#12345](https://github.com/nushell/nushell/pull/12345) | +| [@author](https://github.com/author) | ... | [#12345](https://github.com/nushell/nushell/pull/12345) | ### Enhancing the documentation [[toc](#table-of-content)] + Thanks to all the contributors below for helping us making the documentation of Nushell commands better :pray: -| author | description | url | +| author | description | url | | ------------------------------------ | ----------- | ------------------------------------------------------- | -| [@author](https://github.com/author) | ... | [#12345](https://github.com/nushell/nushell/pull/12345) | +| [@author](https://github.com/author) | ... | [#12345](https://github.com/nushell/nushell/pull/12345) | ## Stricter rules around setting environment variables [[toc](#table-of-content)] @@ -125,6 +239,155 @@ Additionally, we standardize on using records to pass a map of environment varia ## Our set of commands is evolving [[toc](#table-of-content)] ### New commands [[toc](#table-of-content)] + +#### `plugin add` [[toc](#table-of-content)] + +Adds a plugin to the plugin registry file (default `$nu.plugin-path`). + +```nushell +plugin add ~/.cargo/bin/nu_plugin_gstat +``` + +A different registry file can be used if desired: + +```nushell +plugin add --plugin-config ~/polars.msgpackz ~/.cargo/bin/nu_plugin_polars +``` + +Replaces part of `register`, but can be used as a normal command, so it can be used within a script on a generated path. For example the following is now possible: + +```nushell +glob ~/.cargo/bin/nu_plugin_* | each { |file| plugin add $file } +``` + +#### `plugin rm` [[toc](#table-of-content)] + +Removes a plugin from the plugin registry file (default `$nu.plugin-path`). + +```nushell +plugin rm gstat +plugin rm --plugin-config ~/polars.msgpackz polars +``` + +The commands will still remain in scope, but will not be present the next time `nu` is started. + +#### `plugin use` [[toc](#table-of-content)] + +Loads a plugin from the plugin registry file (default `$nu.plugin-path`) **at parse time**. + +This replaces the other part of `register` - actually adding the commands to scope. It does not run the plugin executable, though, it just loads the [previously added](#plugin-add-toc) commands from the registry file. + +#### `from msgpack` [[toc](#table-of-content)] + +Reads [MessagePack](https://msgpack.org) format data into Nu values. + +Example: + +```nushell +> 0x[a7 4e 75 73 68 65 6c 6c] | from msgpack +Nushell + +# Open a MessagePack formatted file: +> open nushell.msgpack +╭─────────┬───────╮ +│ nushell │ rocks │ +╰─────────┴───────╯ +``` + +You can read a stream of multiple MessagePack objects in the same format as the plugin protocol by using `from msgpack --objects`: + +```nushell +> open --raw ~/nu_plugin_example.in.raw | from msgpack --objects +╭───┬─────────────────────────────────────────────────╮ +│ 0 │ ╭───────┬────────────────────────────────────╮ │ +│ │ │ │ ╭──────────┬─────────────────────╮ │ │ +│ │ │ Hello │ │ protocol │ nu-plugin │ │ │ +│ │ │ │ │ version │ 0.93.0 │ │ │ +│ │ │ │ │ │ ╭───┬─────────────╮ │ │ │ +│ │ │ │ │ features │ │ # │ name │ │ │ │ +│ │ │ │ │ │ ├───┼─────────────┤ │ │ │ +│ │ │ │ │ │ │ 0 │ LocalSocket │ │ │ │ +│ │ │ │ │ │ ╰───┴─────────────╯ │ │ │ +│ │ │ │ ╰──────────┴─────────────────────╯ │ │ +│ │ ╰───────┴────────────────────────────────────╯ │ +│ │ │ +│ 1 │ ... │ +│ 2 │ ... │ +│ 3 │ ... │ +│ 4 │ ... │ +│ │ │ +│ 5 │ Goodbye │ +╰───┴─────────────────────────────────────────────────╯ + +``` + +Example captured with [nu_plugin_tracer](https://github.com/tesujimath/nu_plugin_tracer). + +#### `to msgpack` [[toc](#table-of-content)] + +Converts Nu values into [MessagePack](https://msgpack.org). + +Example: + +```nushell +> 'Nushell' | to msgpack +Length: 8 (0x8) bytes | printable whitespace ascii_other non_ascii +00000000: a7 4e 75 73 68 65 6c 6c ×Nushell + +# Save in MessagePack format to nushell.msgpack: +> {nushell: rocks} | save nushell.msgpack +``` + +#### `from msgpackz` [[toc](#table-of-content)] + +Just like [`from msgpack`](#from-msgpack-toc), but decompresses with [brotli](https://github.com/google/brotli) first. + +The `msgpackz` format is used by the new plugin file format: + +```nushell +> open ~/.config/nushell/plugin.msgpackz +╭─────────────────┬─────────────────────────────────────────────────────────────────────────────────╮ +│ nushell_version │ 0.92.3 │ +│ │ ╭───┬───────────────┬────────────────────────────────────┬───────┬────────────╮ │ +│ plugins │ │ # │ name │ filename │ shell │ commands │ │ +│ │ ├───┼───────────────┼────────────────────────────────────┼───────┼────────────┤ │ +│ │ │ 0 │ custom_values │ .cargo/bin/nu_plugin_custom_values │ │ [table 8 │ │ +│ │ │ │ │ │ │ rows] │ │ +│ │ │ 1 │ dbus │ .cargo/bin/nu_plugin_dbus │ │ [table 7 │ │ +│ │ │ │ │ │ │ rows] │ │ +│ │ │ 2 │ emoji │ .cargo/bin/nu_plugin_emoji │ │ [table 1 │ │ +│ │ │ │ │ │ │ row] │ │ +│ │ │ 3 │ example │ .cargo/bin/nu_plugin_example │ │ [table 13 │ │ +│ │ │ │ │ │ │ rows] │ │ +│ │ │ 4 │ explore │ .cargo/bin/nu_plugin_explore │ │ [table 1 │ │ +│ │ │ │ │ │ │ row] │ │ +│ │ │ 5 │ formats │ .cargo/bin/nu_plugin_formats │ │ [table 4 │ │ +│ │ │ │ │ │ │ rows] │ │ +│ │ │ 6 │ gstat │ .cargo/bin/nu_plugin_gstat │ │ [table 1 │ │ +│ │ │ │ │ │ │ row] │ │ +│ │ │ 7 │ inc │ .cargo/bin/nu_plugin_inc │ │ [table 1 │ │ +│ │ │ │ │ │ │ row] │ │ +│ │ ╰───┴───────────────┴────────────────────────────────────┴───────┴────────────╯ │ +╰─────────────────┴─────────────────────────────────────────────────────────────────────────────────╯ +``` + +#### `to msgpackz` [[toc](#table-of-content)] + +Just like [`to msgpack`](#from-msgpack-toc), but compresses with [brotli](https://github.com/google/brotli) after encoding. + +For example, to use it to manually edit the plugin file: + +```nushell +open ~/.config/nushell/plugin.msgpackz | + update plugins { + # Remove plugins with missing files: + where { get filename | path exists } + } | + save -f ~/.config/nushell/plugin.msgpackz +``` + +As with other format commands, `open` and `save` will automatically use the `msgpackz` conversion if the file extension matches. + ### Changes to existing commands [[toc](#table-of-content)] #### `with-env` [[toc](#table-of-content)] @@ -247,8 +510,27 @@ Thanks to [@singh-priyank](https://github.com/singh-priyank) in [#12443](https:/ With [#12432](https://github.com/nushell/nushell/pull/12432), the `type` column returned from `explain` should no longer contain `string` for every row. ### Deprecated commands [[toc](#table-of-content)] + +#### `register` [[toc](#table-of-content)] + +This command is deprecated in favor of the new [`plugin add`](#plugin-add-toc) and [`plugin use`](#plugin-use-toc) commands. See the [relevant section](#redesigned-plugin-management-commands-toc) for the reasoning behind this change. + ### Removed commands [[toc](#table-of-content)] +## For plugin developers [[toc](#table-of-content)] + +### New protocol features [[toc](#table-of-content)] + +- [`LocalSocket`: local socket mode.](/contributor-book/plugin_protocol_reference.md#localsocket-feature) Added in PR [#12448](https://github.com/nushell/nushell/pull/12448). + +### New engine calls [[toc](#table-of-content)] + +See the [Rust docs for `EngineInterface`](https://docs.rs/nu-plugin/0.93.0/nu_plugin/struct.EngineInterface.html) for specific documentation for Rust plugins. + +- [`GetSpanContents`: view the source code of a span.](/contributor-book/plugin_protocol_reference.md#getspancontents-engine-call) Added in PR [#12439](https://github.com/nushell/nushell/pull/12439). +- [`EnterForeground`: enter the terminal controlling (foreground) process group.](/contributor-book/plugin_protocol_reference.md#enterforeground-engine-call) Added in PR [#12448](https://github.com/nushell/nushell/pull/12448). +- [`LeaveForeground`: leave the foreground process group.](/contributor-book/plugin_protocol_reference.md#leaveforeground-engine-call) Added in PR [#12448](https://github.com/nushell/nushell/pull/12448). + # Breaking changes [[toc](#table-of-content)] + # Full changelog [[toc](#table-of-content)] +