Add documentation for meta variable expressions

[c410-f3r]

Requested in rust-lang/rust#122808

[ehuss]

@c410-f3r Can you update this from the changes in rust-lang/rust#124987?

[c410-f3r]

Updated

[tgross35]

Added some suggestions for wording that I think is a bit easier to understand. I think this section could really get its own page since "Macros By Example" is already huge, and metavariable expressions will likely have more entries in the future.

Also, most of this md doc is wrapped to 80 chars, probably good to keep that consistent rather than switching wrapping styles

[tgross35]

Suggested change

The output of `count` depends on where it is placed as well the provided index. If no index is provided, then it will always start at the innermost level.

I would give a simple example first and mention depth after

[RalfJung]

I think this shouldn't be removed, rather it should be expanded to fully describe the exact algorithm used to determine the result. I assume it's something like this but I don't know the right terms for all the concepts involved (and I only experimented with depth=0).

[RalfJung]

In general, it seems to be something like

• let's say $ident occurs under N repetition groups in the matcher
• and now we use ${count($ident, D)} under M repetition groups in the transcriber
• then we must have M+D < N
• and we are counting how often the repetition group D up from $ident in the matcher occurs inside the current iteration of the M outer repetitions in the transcriber. IOW, for the case where D=0 (IMO the only one we should stabilize for now), we are counting how many times $ident would expand if we surrounded $ident in enough repetition groups to make it legal to occur at this point.

[c410-f3r]

In general, it seems to be something like

* let's say `$ident` occurs under N repetition groups in the matcher

* and now we use `${count($ident, D)}` under M repetition groups in the transcriber

* then we must have M+D < N

* and we are counting how often the repetition group D up from `$ident` in the matcher occurs inside the current iteration of the M outer repetitions in the transcriber. IOW, for the case where D=0 (IMO the only one we should stabilize for now), we are counting how many times `$ident` would expand  if we surrounded `$ident` in enough repetition groups to make it legal to occur at this point.

That is a good explanation. Specially in the "counting how often the repetition group D up from $ident in the matcher occurs" highlighting that $ident is just a matcher reference when using indexes.

[tgross35]

Suggested change

	macro_rules! no_repetition0 {
	( $( $a:ident: $( $b:literal ),* );+ ) => {
	[${count($b)}]
	};
	}
	macro_rules! no_repetition1 {
	( $( $a:ident: $( $b:literal ),* );+ ) => {
	[${count($b, 1)}]
	};
	}
	macro_rules! outermost {
	( $( $a:ident: $( $b:literal ),* );+ ) => {
	[$( ${ignore($a)} ${count($b)}, )+]
	};
	}
	fn main() {
	// 1 2 3 4 5 = 5 elements
	assert_eq!(no_repetition0!(a: 1, 2, 3; b: 4, 5), [5]);
	// a b = 2 elements
	assert_eq!(no_repetition1!(a: 1, 2, 3; b: 4, 5), [2]);
	// 1 2 3 = 3 elements
	// 4 5 = 2 elements
	assert_eq!(outermost!(a: 1, 2, 3; b: 4, 5), [3, 2]);
	}
	macro_rules! count_value {
	( $( $name:ident: $( $value:literal ),* );+ ) => {
	// Count the total number of times that the (innermost) group
	// containing `$value` gets matched.
	${count($value)}
	// This is the same as `${count($value, 0)}`
	};
	}
	macro_rules! count_name1 {
	( $( $name:ident: $( $value:literal ),* );+ ) => {
	// This is one way to get the number of times that the group
	// containing `$name` gets matched. Alternatively...
	${count($name)}
	};
	}
	macro_rules! count_name2 {
	( $( $name:ident: $( $value:literal ),* );+ ) => {
	// ...`$value` can be used again with a depth specifier of 1,
	// indicating that repetitions of the 1st parent group of
	// `$value` should be counted, rather than `$value`'s innermost group.
	//
	// `1` is the maximum value here since `$value`'s group has a single
	// parent.
	${count($value, 1)}
	};
	}
	macro_rules! count_value_nested {
	( $( $name:ident: $( $value:literal ),* );+ ) => {
	[ $(
	// using `count` within a repetition group will return the number
	// of times `$value` is matched _within that group_.
	${count($value)},
	)+ ]
	};
	}
	fn main() {
	// all instances of `$value` counted: count(1, 2, 3, 4, 5) = 5
	assert_eq!(count_value!(a: 1, 2, 3; b: 4, 5), 5);
	// count(1, 2, 3, ... 11) = 11
	assert_eq!(
	count_value!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
	11
	);
	// `$value` is never matched; count() = 0
	assert_eq!(count_value!(a:), 0);
	// count(a, b) = 2 matches
	assert_eq!(count_name1!(a: 1, 2, 3; b: 4, 5), 2);
	// count(a, b, c, d) = 4 matches
	assert_eq!(
	count_name1!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
	4
	);
	// count(a) = 1 match
	assert_eq!(count_name1!(a:), 1);
	// These have the same results as the above
	assert_eq!(count_name2!(a: 1, 2, 3; b: 4, 5), 2);
	assert_eq!(
	count_name2!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
	4
	);
	assert_eq!(count_name2!(a:), 1);
	// first match: count(1, 2, 3) = 3. second match: count(4, 5) = 2.
	assert_eq!(count_value_nested!(a: 1, 2, 3; b: 4, 5), [3, 2]);
	// first match: count(1, 2, 3) = 3. second match: count(4, 5) = 2.
	// third match: count(6, 7) = 4. fourth match: count(8, 9, 10, 11) = 4.
	assert_eq!(
	count_value_nested!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
	[3, 2, 2, 4]
	);
	// `$value` is never matched; count() = 0
	assert_eq!(count_value_nested!(a:), [0]);
	}

• Added guiding comments
• Change $a and $b matcher names, which I found confusing since they don't align with the a and b identifiers within macro usage blocks.
• ignore isn't needed here
• Add some more examples

[RalfJung]

using count within a repetition group will return the number>
of times $value is matched within that group.

This is a key part of the spec. It should be in the reference text above, not just in the examples.

[RalfJung]

Would be good to have an example of that; I don't know what is meant here.

Is this even about the innermost repetition? In ( $( $name:ident: $( $value:literal ),* );+ ), can I do $( ${count($name) }? That is not the innermost repetition, as value is even-more-inner -- but I am trying to count name inside a repetition for name which will always return 1.

[c410-f3r]

Is this even about the innermost repetition?

That is correct, count can not be placed inside the repetition depth of its referenced meta-variable. So innermost in regards to the macro declaration is inaccurate.

can I do $( ${count($name) }?

No because of the above explanation.

but I am trying to count name inside a repetition for name which will always return 1.

That is the reason behind such constraint. It doesn't make sense to count something that will always yield 1.

Nevertheless, it can be relaxed in the future if so desired.

[traviscross]

@c410-f3r: We looked at this in the rustdocs call today. Do you have any thoughts on the feedback given above? If so, that could help us in reviewing this.

[c410-f3r]

Feedback looks good but I personally prefer to wait for the decision about rust-lang/rust#122808 (comment).

Otherwise this could be my third reverted PR. See #1179, #1251, #1192 and rust-lang/rust#99435.

[RalfJung]

Feedback looks good but I personally prefer to wait for the decision about rust-lang/rust#122808 (comment).

The problem is that currently we are guessing what the proposed semantics even are. So someone who knows the implementation needs to write up-to-date information on that before a decision can be made. Some of the feedback in this PR is of the form "we still don't know what the semantics are, please clarify".

This is why we generally require a reference PR before t-lang can make a final stabilization decision: they need to know what is even proposed for stabilization!

[c410-f3r]

I will try to address all questions this weekend in my free time.

For what it is worth, here goes another illustrative example in the meanwhile.

#![feature(macro_metavar_expr)]

macro_rules! no_repetition {
    ( $( [ $( ( $($i:ident)* ) )* ] )* ) => {
        (
            // ****** 1 `ident` count *****
            //
            // Guided by 6 innermost `ident` from `$($i:ident)*`
            //
            // 6 (a b c d e f)
            [${count($i)}],

            // ****** 1 `ident` count *****
            //
            // Guided by 5 middle `(...)` from `$( ( $($i:ident)* ) )*`
            //
            // 5 (() () () () ())
            [${count($i, 1)}],

            // ****** 1 `ident` count *****
            //
            // Guided by 3 outermost `[...]` from `$( [ $( ( $($i:ident)* ) )* ] )*`
            //
            // 3 ([] [] [])
            [${count($i, 2)}],
        )
    }
}

macro_rules! one_repetition {
    ( $( [ $( ( $($i:ident)* ) )* ] )* ) => {
        (
            // ****** 3 `[...]` counts *****
            //
            // Guided by 6 innermost `ident` from `$($i:ident)*`
            //
            // 2 (a b)
            // 4 (c d e f)
            // 0
            [$( ${count($i)}, )*],

            // ****** 3 `[...]` counts *****
            //
            // Guided by 5 middle `(...)` from `$( ( $($i:ident)* ) )*`
            //
            // 2 (() ())
            // 3 (() () ())
            // 0
            [$( ${count($i, 1)}, )*],
        )
    }
}

macro_rules! two_repetitions {
    ( $( [ $( ( $($i:ident)* ) )* ] )* ) => {
        (
            // ****** 5 `(...)` counts *****
            //
            // Guided by 6 innermost `ident` from `$($i:ident)*`
            //
            // 2 (a b)
            // 0
            // 1 (c)
            // 0
            // 3 (d e f)
            [$( $( ${count($i)}, )* )*],
        )
    }
}

fn main() {
    let (innermost, middle, outermost) = no_repetition!([(a b) ()] [(c) () (d e f)] []);
    assert_eq!(innermost, [6]);
    assert_eq!(middle, [5]);
    assert_eq!(outermost, [3]);

    let (innermost, middle) = one_repetition!([(a b) ()] [(c) () (d e f)] []);
    assert_eq!(innermost, [2, 4, 0]);
    assert_eq!(middle, [2, 3, 0]);

    let (innermost,) = two_repetitions!([(a b) ()] [(c) () (d e f)] []);
    assert_eq!(innermost, [2, 0, 1, 0, 3]);
}