POC of Approach to Supporting Internationalization #5853

toadslop · 2024-12-22T05:58:34Z

Support for Internationalization

Issue: #380

Provides a POC demonstrating a method for adding i18n support to Clap in a minimally invasive way. This POC is not a complete solution and only aims to illustrate the approach for evaluation of the project's maintainers. Additional work will be needed to produce a production-ready solution.

General Notes

For context, this approach is not an internationlization solution. Rather, it is a solution which extracts the hardcoded English text of the library into external configuration files, and then offers an API for consumers to replace the texts with their own. It does not care about the content of the texts, giving the consumer the freedom to substitute content for any reason and under any conditions they choose; only one such use of this is for internationalization.

Trying It Out

To get a quick look at the results, you can run the accompanying example. You'll note that the alignment of the of the columns is a bit off -- this is a consequence of the current POC implementation. If the general approach is accepted, then I will consider how to resolve this.

gh pr checkout <TODO>

cargo run --example i18n --features derive -- --help
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/examples/i18n --help`
A simple to use, efficient, and full-featured Command Line Argument Parser

Usage: i18n [OPTIONS] [NAME] [COMMAND]

Commands:
  test  Does testing things
  help  Print this message or the help of the given subcommand(s)

Arguments:
  [NAME]  Please enter your name

Options:
  -c, --config <FILE>  Enter the path to a configuration file
  -d, --debug...         Execute in debug mode
  -h, --help             Print help
  -V, --version          Print version

LOCALE=jp cargo run --example i18n --features derive -- --help
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.08s
     Running `target/debug/examples/i18n --help`
A simple to use, efficient, and full-featured Command Line Argument Parser

使用方法: i18n [オプション] [名前] [コマンド]

コマンド:
  test  テストを実行します
  help  このメッセージまたは指定されたサブコマンドのヘルプを表示します

引数:
  [名前]  名前を入力してください

オプション:
  -c, --config <ファイル>  設定ファイルのパスを入力してください
  -d, --debug...         デバッグモードで実行します
  -h, --help             ヘルプを表示
  -V, --version          バージョンを表示

Approach

The main design goal of this implementation is to be minimally invasive. By minimally invasive, I mean that I do not want to significantly impact how Clap already works.

Current Behavior

Clap currently produces output in the following stages:

load a layout template
interpolate hardcoded text and user provided text into the template while applying styling
print output to terminal (or otherwise, make the output available to the consumer)

This implementation changes step 2 and adds a new step between it and step 3.

load a layout template
interpolate hardcoded keys and user-provided keys into the layout template, producing a new template file which is styled but does not include text -- instead it includes the text-keys surrounded by template syntax (current something lie {clap.default-help-text}
run a pass over the template from step two to interpolate the actual texts
print output to terminal (or otherwise, make the output available to the consumer)

Note that the current implementation tends to break column alignments since the widths of the text keys are often different from the actual text, and it appears that Step 2 aligns the columes based on the input text. Some consideration will be needed to determine how best to resolve this issue but this can be deferred until I get the go ahead from the Clap team.

**Problem:** Visible alias completions were causing duplicate case conditions when offering subcommand completions, due to the fact that they share the same function name as the command they are aliasing. **Solution:** Deduplicate them so as to not redefine the exact same case statement logic.

toadslop · 2024-12-22T05:59:19Z

Cargo.toml

@@ -189,6 +189,8 @@ snapbox = "0.6.16"
 shlex = "1.3.0"


Just adding a new example to demonstrate, and some dev-deps to support the example.

toadslop · 2024-12-22T06:01:47Z

clap_bench/benches/complex.rs

@@ -1,6 +1,6 @@
 #![allow(elided_lifetimes_in_paths)] // needed for divan



You'll see a lot of cases of text_provider::DEFAULT_TEXT_PROVIDER being added throughout this PR. This is just to resolve compile issues. The final implementation would ideally be structured in a way to avoid needing to make these changes; however, to keep the POC simple I opted to not resolve that issue until later.

toadslop · 2024-12-22T06:02:28Z

clap_builder/Cargo.toml

@@ -67,6 +67,8 @@ anstyle = "1.0.8"
 terminal_size = { version = "0.4.0", optional = true }
 backtrace = { version = "0.3.73", optional = true }
 unicode-width = { version = "0.2.0", optional = true }


New depts. serde_yml is for deserializing the external config file where the hardcoded texts are extracted to.

toadslop · 2024-12-22T06:03:34Z

clap_builder/src/builder/command.rs

@@ -4729,7 +4734,7 @@ impl Command {
                    .help("Print help (see more with '--help')")
                    .long_help("Print help (see a summary with '-h')");
            } else {


This is an example of one case where we have removed hardcoded English text and replaced it with a template string containing a text lookup key.

toadslop · 2024-12-22T06:05:11Z

clap_builder/src/builder/styled_str.rs

@@ -29,6 +31,11 @@ impl StyledStr {
        Self(String::new())
    }



This new interpolate function replaces the text lookup keys with the actual texts. There is probably a better place to put this, but for the purposes of demonstrating the approach, I thought this was good enough.

toadslop · 2024-12-22T06:07:35Z

clap_builder/src/derive.rs

@@ -28,15 +29,22 @@ use std::ffi::OsString;
 pub trait Parser: FromArgMatches + CommandFactory + Sized {
    /// Parse from `std::env::args_os()`, [exit][Error::exit] on error.
    fn parse() -> Self {
-        let mut matches = <Self as CommandFactory>::command().get_matches();


New parse_with_texts is the API which allows a consumer to substitute the OOTB English texts with their own texts. This could perhaps be adjusted to allow a reference to the texts to be added to the Parser struct itself so that we don't need to pass them down through many layers of function calls as we do here. This change might have required making changes to clap_derive in order to get a minimally working example ready, so I decided against it.

toadslop · 2024-12-22T06:08:21Z

clap_builder/src/output/help_template.rs

@@ -198,7 +198,7 @@ impl<'cmd, 'writer> HelpTemplate<'cmd, 'writer> {
                    "usage-heading" => {
                        let _ = write!(
                            self.writer,


More hardcoded texts replaced with text lookup keys.

toadslop · 2024-12-22T06:09:19Z

clap_builder/src/text_provider.rs

@@ -0,0 +1,48 @@
+//! Utilities for loading help texts from external sources.


Simple implementation of what the API for allowing consumers to pass in their own texts might look like.

toadslop · 2024-12-22T06:11:36Z

clap_builder/src/util/template.rs

@@ -0,0 +1,90 @@
+use crate::text_provider::TextProvider;


Simple template engine for interpolating the actual texts into the templates using the text lookup keys. Hand-rolled this to avoid pulling in a whole templating engine, which would include many more features that what we actually need for this case.

Some improvements to this might be to allow escaping {. Also would need some tests since this hastilly written example might have a few bugs lurking in it. But again, this is just a minimal example for the POC.

toadslop · 2024-12-22T06:12:55Z

clap_builder/texts/en.yaml

@@ -0,0 +1,10 @@
+clap.usage.header-text: Usage


These are the hard-coded default texts that I extracted. You'll note that there are many more texts than these -- I just extracted enough to demonstrate using the example that I included in this PR.

toadslop · 2024-12-22T06:15:17Z

examples/i18n/main.rs

+use rust_i18n::Backend;
+use std::path::PathBuf;
+use clap::{Parser, Subcommand};
+


For this example, we use the rust-i18n crate to demonstrate how we can use the TextProvider API to achieve internationalization. I chose this crate because it allows looking translations at runtime, which is probably what most people would want since it avoids packaging languages into the binary that the end user will never use.

toadslop · 2024-12-22T06:16:54Z

examples/i18n/main.rs

+#[derive(Parser)]
+#[command(version, about, long_about = None)]
+struct Cli {
+    /// {name-help-text}


Notice that we need to provide a template string with the text lookup key inside in order to have Clap interpolate the text. This may not be ideal, but it was the simplest implementation as it didn't require making any changes to clap_derive.

toadslop · 2024-12-22T06:17:23Z

examples/i18n/main.rs

+}
+
+fn main() {
+    let locale = env::var("LOCALE").unwrap_or("en".into());


Allow the user to change the locale using an environment variable.

toadslop · 2024-12-22T06:18:41Z

examples/i18n/main.rs

+    let cli = Cli::parse_with_texts(&I18n);
+    // You can check the value provided by positional arguments, or option arguments
+    if let Some(name) = cli.name.as_deref() {
+        println!("Value for name: {name}");


The user would be free to continue using rust-i18n to internationalize this and the following printlines as well.

toadslop · 2024-12-22T06:20:06Z

tests/builder/app_settings.rs

@@ -137,6 +137,9 @@ fn arg_required_else_help_with_default() {
    );
 }



This test shows a weakness of the current simple implementation -- the end user is able to extract the text from the error message without calling the print method, which is where I currently carry out interpolation. Will need to find a more ideal place to put that logic so that users can never extract text from Clap that hasn't been interpolated.

ribru17 and others added 6 commits December 21, 2024 17:21

docs: Update changelog

75af6df

chore: Release

a005f66

adds infrastructure for extracing hardcoded texts

70f6629

pass in default text provide to existing examples and tests

fa3e506

add example

d2435e9

toadslop commented Dec 22, 2024

View reviewed changes

toadslop mentioned this pull request Dec 22, 2024

Support or document localization of messages and flags #380

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

POC of Approach to Supporting Internationalization #5853

POC of Approach to Supporting Internationalization #5853

toadslop commented Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

toadslop Dec 22, 2024

		@@ -1,6 +1,6 @@
		#![allow(elided_lifetimes_in_paths)] // needed for divan

		@@ -0,0 +1,48 @@
		//! Utilities for loading help texts from external sources.

		@@ -137,6 +137,9 @@ fn arg_required_else_help_with_default() {
		);
		}

POC of Approach to Supporting Internationalization #5853

Are you sure you want to change the base?

POC of Approach to Supporting Internationalization #5853

Conversation

toadslop commented Dec 22, 2024

Support for Internationalization

General Notes

Trying It Out

Approach

Current Behavior

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment