Reduce formatting `width` and `precision` to 16 bits #136932

m-ou-se · 2025-02-12T16:46:32Z

This is part of #99012

This is reduces the width and precision fields in format strings to 16 bits. They are currently full usizes, but it's a bit nonsensical that we need to support the case where someone wants to pad their value to eighteen quintillion spaces and/or have eighteen quintillion digits of precision.

By reducing these fields to 16 bit, we can reduce FormattingOptions to 64 bits (see #136974) and improve the in memory representation of format_args!(). (See additional context below.)

This also fixes a bug where the width or precision is silently truncated when cross-compiling to a target with a smaller usize. By reducing the width and precision fields to the minimum guaranteed size of usize, 16 bits, this bug is eliminated.

This is a breaking change, but affects almost no existing code.

Details of this change:

There are three ways to set a width or precision today:

Directly a formatting string, e.g. println!("{a:1234}")
Indirectly in a formatting string, e.g. println!("{a:width$}", width=1234)
Through the unstable FormattingOptions::width method.

This PR:

Adds a compiler error for 1. (println!("{a:9999999}") no longer compiles and gives a clear error.)
Adds a runtime check for 2. (println!("{a:width$}, width=9999999) will panic.)
Changes the signatures of the (unstable) FormattingOptions::[get_]width methods to use a u16 instead.

Additional context for improving FormattingOptions and fmt::Arguments:

All the formatting flags and options are currently:

The + flag (1 bit)
The - flag (1 bit)
The # flag (1 bit)
The 0 flag (1 bit)
The x? flag (1 bit)
The X? flag (1 bit)
The alignment (2 bits)
The fill character (21 bits)
Whether a width is specified (1 bit)
Whether a precision is specified (1 bit)
If used, the width (a full usize)
If used, the precision (a full usize)

Everything except the last two can simply fit in a u32 (those add up to 31 bits in total).

If we can accept a max width and precision of u16::MAX, we can make a FormattingOptions that is exactly 64 bits in size; the same size as a thin reference on most platforms.

If, additionally, we also limit the number of formatting arguments, we can also reduce the size of fmt::Arguments (that is, of a format_args!() expression).

m-ou-se · 2025-02-12T16:48:10Z

@bors try

bors · 2025-02-12T16:49:22Z

⌛ Trying commit 7355738 with merge 7af7790...

…try> Reduce formatting `width` and `precision` to 16 bits This is reduces the `width` and `precision` fields in format strings to 16 bits. They are currently full `usize`s, but it's a bit nonsensical that we need to support the case where someone wants to pad their value to eighteen quintillion spaces and/or have eighteen quintillion digits of precision. This is a breaking change, but probably affects virtually no code. Let's do a crater run to find out. Marking this as experiment for now. --- There are three ways to set a width or precision today: 1. Directly a formatting string, e.g. `println!("{a:1234}")` 2. Indirectly in a formatting string, e.g. `println!("{a:width$}", width=1234)` 3. Through the unstable `FormattingOptions::width` method. This PR: - Adds a compiler error for 1. (`println!("{a:9999999}")` no longer compiles and gives a clear error.) - Adds a runtime check for 2. (`println!("{a:width$}, width=9999999)` will panic.) - Changes the signature of `FormattingOptions::width` to take a `u16` instead. --- Additional context: All the formatting flags and options are currently: - The `+` flag (1 bit) - The `-` flag (1 bit) - The `#` flag (1 bit) - The `0` flag (1 bit) - The `x?` flag (1 bit) - The `X?` flag (1 bit) - The alignment (2 bits) - The fill character (21 bits) - Whether a width is specified (1 bit) - Whether a precision is specified (1 bit) - If used, the width (a full usize) - If used, the precision (a full usize) Everything except the last two can simply fit in a `u32` (those add up to 31 bits in total). If we can accept a max width and precision of u16::MAX, we can make a `FormattingOptions` that is exactly 64 bits in size; the same size as a thin reference on most platforms. If, additionally, we also limit the number of formatting arguments to u16::MAX, we can also reduce the size of `fmt::Arguments` (that is, of a `format_args!()` expression).

tests/mir-opt/funky_arms.float_to_exponential_common.GVN.panic-abort.diff

bors · 2025-02-12T18:41:27Z

☀️ Try build successful - checks-actions
Build commit: 7af7790 (7af779037716ae4125ceabb429791b4cf5dd0a43)

m-ou-se · 2025-02-12T19:16:27Z

@craterbot run mode=build-and-test

craterbot · 2025-02-12T19:16:55Z

👌 Experiment pr-136932 created and queued.
🤖 Automatically detected try build 7af7790
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

craterbot · 2025-02-12T19:17:23Z

🚧 Experiment pr-136932 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

oxalica · 2025-02-13T01:39:12Z

This is a breaking change, but probably affects virtually no code. Let's do a crater run to find out. Marking this as experiment for now.

As a data point, I do use usize width for indentation generation of pretty-printer in some of my crates.
It's used as a shortcut for " ".repeat(width) without allocation (at least I think so?). I can accept to enforce u16 here because nobody actually use 65536 indentation levels, but usize is definitely a more natural choice for "length" semantic than u16.

write!(self.w, "\n{:1$}]", "", self.cur_indent)?;
write!(f, "{:len$}", "", len = (stride - 1) * 4)?;

m-ou-se · 2025-02-13T09:44:23Z

The type will stay a usize; changing that would break a lot of things. This PR adds a runtime check (panic) that the usize you give it isn't above u16::MAX. (And then stores everything internally as a u16.)

Reduce FormattingOptions to 64 bits This reduces FormattingOptions from 6-7 machine words (384 bits on 64-bit platforms, 224 bits on 32-bit platforms) to just 64 bits (a single register on 64-bit platforms). This PR includes rust-lang#136932, which reduces the width and precision options to 16 bits, to make it all fit. Before: ```rust pub struct FormattingOptions { flags: u32, // only 6 bits used fill: char, align: Option<Alignment>, width: Option<usize>, precision: Option<usize>, } ``` After: ```rust pub struct FormattingOptions { /// Bits: /// - 0: `+` flag [rt::Flag::SignPlus] /// - 1: `-` flag [rt::Flag::SignMinus] /// - 2: `#` flag [rt::Flag::Alternate] /// - 3: `0` flag [rt::Flag::SignAwareZeroPad] /// - 4: `x?` flag [rt::Flag::DebugLowerHex] /// - 5: `X?` flag [rt::Flag::DebugUpperHex] /// - 6-7: Alignment (0: Left, 1: Right, 2: Center, 3: Unknown) /// - 8: Width flag (if set, the width field below is used) /// - 9: Precision flag (if set, the precision field below is used) /// - 10: unused /// - 11-31: fill character (21 bits, a full `char`) flags: u32, /// Width if width flag above is set. Otherwise, always 0. width: u16, /// Precision if precision flag above is set. Otherwise, always 0. precision: u16, } ```

craterbot · 2025-02-13T22:15:07Z

🎉 Experiment pr-136932 is completed!
📊 172 regressed and 113 fixed (582049 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

m-ou-se · 2025-02-14T08:35:30Z

Crater results: Only two regressions are caused by an error or panic from this change.

~~gh/vasekp/stream-rust - Runtime panic - Using a value close to usize::MAX as precision~~ - Update: fixed
~~gh/uutils/coreutils - Compiler error in a unit test~~ - Update: fixed

scottmcm · 2025-03-10T17:06:53Z

@bors r+ rollup=iffy (might want to bisect here)

bors · 2025-03-10T17:06:56Z

📌 Commit 2ce0205 has been approved by scottmcm

It is now in the queue for this repository.

Feedback was addressed.

…=scottmcm Reduce formatting `width` and `precision` to 16 bits This is part of rust-lang#99012 This is reduces the `width` and `precision` fields in format strings to 16 bits. They are currently full `usize`s, but it's a bit nonsensical that we need to support the case where someone wants to pad their value to eighteen quintillion spaces and/or have eighteen quintillion digits of precision. By reducing these fields to 16 bit, we can reduce `FormattingOptions` to 64 bits (see rust-lang#136974) and improve the in memory representation of `format_args!()`. (See additional context below.) This also fixes a bug where the width or precision is silently truncated when cross-compiling to a target with a smaller `usize`. By reducing the width and precision fields to the minimum guaranteed size of `usize`, 16 bits, this bug is eliminated. This is a breaking change, but affects almost no existing code. --- Details of this change: There are three ways to set a width or precision today: 1. Directly a formatting string, e.g. `println!("{a:1234}")` 2. Indirectly in a formatting string, e.g. `println!("{a:width$}", width=1234)` 3. Through the unstable `FormattingOptions::width` method. This PR: - Adds a compiler error for 1. (`println!("{a:9999999}")` no longer compiles and gives a clear error.) - Adds a runtime check for 2. (`println!("{a:width$}, width=9999999)` will panic.) - Changes the signatures of the (unstable) `FormattingOptions::[get_]width` methods to use a `u16` instead. --- Additional context for improving `FormattingOptions` and `fmt::Arguments`: All the formatting flags and options are currently: - The `+` flag (1 bit) - The `-` flag (1 bit) - The `#` flag (1 bit) - The `0` flag (1 bit) - The `x?` flag (1 bit) - The `X?` flag (1 bit) - The alignment (2 bits) - The fill character (21 bits) - Whether a width is specified (1 bit) - Whether a precision is specified (1 bit) - If used, the width (a full usize) - If used, the precision (a full usize) Everything except the last two can simply fit in a `u32` (those add up to 31 bits in total). If we can accept a max width and precision of u16::MAX, we can make a `FormattingOptions` that is exactly 64 bits in size; the same size as a thin reference on most platforms. If, additionally, we also limit the number of formatting arguments, we can also reduce the size of `fmt::Arguments` (that is, of a `format_args!()` expression).

Rollup of 16 pull requests Successful merges: - rust-lang#126856 (remove deprecated tool `rls`) - rust-lang#136932 (Reduce formatting `width` and `precision` to 16 bits) - rust-lang#137314 (change definitely unproductive cycles to error) - rust-lang#137612 (Update bootstrap to edition 2024) - rust-lang#138002 (Disable CFI for weakly linked syscalls) - rust-lang#138052 (strip `-Wlinker-messages` wrappers from `rust-lld` rmake test) - rust-lang#138063 (Improve `-Zunpretty=hir` for parsed attrs) - rust-lang#138109 (make precise capturing args in rustdoc Json typed) - rust-lang#138147 (Add maintainers for powerpc64le-unknown-linux-gnu) - rust-lang#138245 (stabilize `ci_rustc_if_unchanged_logic` test for local environments) - rust-lang#138296 (Remove `AdtFlags::IS_ANONYMOUS` and `Copy`/`Clone` condition for anonymous ADT) - rust-lang#138300 (add tracking issue for unqualified_local_imports) - rust-lang#138307 (Allow specifying glob patterns for try jobs) - rust-lang#138313 (Update books) - rust-lang#138315 (use next_back() instead of last() on DoubleEndedIterator) - rust-lang#138318 (Rustdoc: remove a bunch of `@ts-expect-error` from main.js) r? `@ghost` `@rustbot` modify labels: rollup

bors · 2025-03-11T04:07:09Z

⌛ Testing commit 2ce0205 with merge 374ce1f...

matthiaskrgr · 2025-03-11T05:59:30Z

@jieyouxu

this means we should NOT burry this in a rollup of 25 PRs ..

jieyouxu · 2025-03-11T06:43:25Z

this means we should NOT burry this in a rollup of 25 PRs ..

If this is perf-sensitive, then it should be marked as rollup=never, or it can be perf-build specifically post rollup. The size of the rollup has no bearing to this, you would have to perf build separately in a 5 PR rollup anyway.

jieyouxu · 2025-03-11T07:00:56Z

For future bisection,
@bors rollup=never

bors · 2025-03-11T07:11:57Z

☀️ Test successful - checks-actions
Approved by: scottmcm
Pushing 374ce1f to master...

rust-log-analyzer · 2025-03-11T07:13:00Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

   Compiling clap v4.5.28
   Compiling build_helper v0.1.0 (/home/runner/work/rust/rust/src/build_helper)
   Compiling citool v0.1.0 (/home/runner/work/rust/rust/src/ci/citool)
    Finished `release` profile [optimized] target(s) in 33.53s
     Running `target/release/citool post-merge-analysis 90384941aae4ea38de00e4702b50757e9b882a19 374ce1f90951b4dd1c8789c6a5905abe8ea99ef8`
error: unrecognized subcommand 'post-merge-analysis'

  tip: a similar subcommand exists: 'post-merge-report'

Usage: citool <COMMAND>

For more information, try '--help'.
##[error]Process completed with exit code 2.

rust-timer · 2025-03-11T08:28:52Z

Finished benchmarking commit (374ce1f): comparison URL.

Overall result: ❌ regressions - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

If the regression was expected or you think it can be justified,
please write a comment with sufficient written justification, and add
@rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
If you think that you know of a way to resolve the regression, try to create
a new PR with a fix for the regression.
If you do not understand the regression or you think that it is just noise,
you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.5%]	3
Regressions ❌ (secondary)	1.2%	[1.2%, 1.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.4%	[0.4%, 0.5%]	3

Max RSS (memory usage)

Results (primary -1.1%, secondary 2.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.1%, 2.4%]	2
Regressions ❌ (secondary)	2.8%	[2.1%, 4.1%]	6
Improvements ✅ (primary)	-7.8%	[-7.8%, -7.8%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-1.1%	[-7.8%, 2.4%]	3

Cycles

Results (secondary -1.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	5.3%	[5.3%, 5.4%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.8%	[-4.9%, -2.2%]	6
All ❌✅ (primary)	-	-	0

Binary size

Results (primary -0.0%, secondary 0.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.0%	[0.0%, 0.2%]	32
Regressions ❌ (secondary)	0.0%	[0.0%, 0.0%]	3
Improvements ✅ (primary)	-0.1%	[-0.5%, -0.1%]	10
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.0%	[-0.5%, 0.2%]	42

Bootstrap: 782.162s -> 782.387s (0.03%)
Artifact size: 365.19 MiB -> 365.21 MiB (0.00%)

m-ou-se added T-libs Relevant to the library team, which will review and decide on the PR/issue. A-fmt Area: `core::fmt` S-experimental Status: Ongoing experiment that does not require reviewing and won't be merged in its current state. labels Feb 12, 2025

m-ou-se self-assigned this Feb 12, 2025

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 12, 2025

m-ou-se mentioned this pull request Feb 12, 2025

core: use FormattingOptions by reference #136862

Closed

m-ou-se removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 12, 2025

scottmcm reviewed Feb 12, 2025

View reviewed changes

tests/mir-opt/funky_arms.float_to_exponential_common.GVN.panic-abort.diff Show resolved Hide resolved

This comment has been minimized.

Sign in to view

craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-experimental Status: Ongoing experiment that does not require reviewing and won't be merged in its current state. labels Feb 12, 2025

m-ou-se force-pushed the fmt-width-precision-u16 branch from 7355738 to ccb9429 Compare February 12, 2025 20:05

m-ou-se mentioned this pull request Feb 13, 2025

Reduce FormattingOptions to 64 bits #136974

Draft

craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Feb 13, 2025

m-ou-se added I-libs-nominated Nominated for discussion during a libs team meeting. I-libs-api-nominated Nominated for discussion during a libs-api team meeting. labels Feb 14, 2025

m-ou-se marked this pull request as ready for review February 14, 2025 08:45

m-ou-se added 6 commits March 10, 2025 12:20

Limit formatting width and precision to 16 bits.

fb9ce02

Update tests.

4374d54

Fix rust-analyzer for 16-bit fmt width and precision.

ce512c2

Remove unnecessary semicolon.

7677567

Add #[track_caller] to from_usize.

2647cf1

Share implementation of expr_u{16,32,size}.

2ce0205

m-ou-se force-pushed the fmt-width-precision-u16 branch from 1fe3f1b to 2ce0205 Compare March 10, 2025 12:58

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 10, 2025

jieyouxu mentioned this pull request Mar 10, 2025

Rollup of 25 pull requests #138324

Closed

jieyouxu mentioned this pull request Mar 10, 2025

Rollup of 17 pull requests #138325

Closed

jieyouxu mentioned this pull request Mar 11, 2025

Rollup of 16 pull requests #138326

Closed

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 11, 2025

bors merged commit 374ce1f into rust-lang:master Mar 11, 2025
7 checks passed

rustbot added this to the 1.87.0 milestone Mar 11, 2025

rustbot added the perf-regression Performance regression. label Mar 11, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce formatting `width` and `precision` to 16 bits #136932

Reduce formatting `width` and `precision` to 16 bits #136932

m-ou-se commented Feb 12, 2025 •

edited

Loading

m-ou-se commented Feb 12, 2025

bors commented Feb 12, 2025

This comment has been minimized.

bors commented Feb 12, 2025

m-ou-se commented Feb 12, 2025

craterbot commented Feb 12, 2025

craterbot commented Feb 12, 2025

oxalica commented Feb 13, 2025

m-ou-se commented Feb 13, 2025 •

edited

Loading

craterbot commented Feb 13, 2025

m-ou-se commented Feb 14, 2025 •

edited

Loading

scottmcm commented Mar 10, 2025

bors commented Mar 10, 2025

bors commented Mar 11, 2025

matthiaskrgr commented Mar 11, 2025

jieyouxu commented Mar 11, 2025

jieyouxu commented Mar 11, 2025

bors commented Mar 11, 2025

rust-log-analyzer commented Mar 11, 2025

rust-timer commented Mar 11, 2025

Reduce formatting width and precision to 16 bits #136932

Reduce formatting width and precision to 16 bits #136932

Conversation

m-ou-se commented Feb 12, 2025 • edited Loading

m-ou-se commented Feb 12, 2025

bors commented Feb 12, 2025

This comment has been minimized.

bors commented Feb 12, 2025

m-ou-se commented Feb 12, 2025

craterbot commented Feb 12, 2025

craterbot commented Feb 12, 2025

oxalica commented Feb 13, 2025

m-ou-se commented Feb 13, 2025 • edited Loading

craterbot commented Feb 13, 2025

m-ou-se commented Feb 14, 2025 • edited Loading

scottmcm commented Mar 10, 2025

bors commented Mar 10, 2025

bors commented Mar 11, 2025

matthiaskrgr commented Mar 11, 2025

jieyouxu commented Mar 11, 2025

jieyouxu commented Mar 11, 2025

bors commented Mar 11, 2025

rust-log-analyzer commented Mar 11, 2025

rust-timer commented Mar 11, 2025

Overall result: ❌ regressions - please read the text below

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Reduce formatting `width` and `precision` to 16 bits #136932

Reduce formatting `width` and `precision` to 16 bits #136932

m-ou-se commented Feb 12, 2025 •

edited

Loading

m-ou-se commented Feb 13, 2025 •

edited

Loading

m-ou-se commented Feb 14, 2025 •

edited

Loading