Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Null-forgiving operator #1195

Draft
wants to merge 5 commits into
base: draft-v8
Choose a base branch
from
Draft

Conversation

Nigel-Ecma
Copy link
Contributor

Null-forgiving operator

  • Fix grammar
  • Specify semantic restrictions
  • Add to precedence table
  • Add to MLR group

Addresses issue #1190

- Fix grammar
- Specify semantic restrictions
- Add to precedence table
- Add to MLR group
- Fix grammar
- Specify semantic restrictions
- Add to precedence table
- Add to MLR group
@Nigel-Ecma
Copy link
Contributor Author

Note: The grammar validation will fail until the validator is updated. The validator is a work-in-progress and there is a mutual dependency between that and this PR. The PR for the validator will follow in the not too distant future…
@BillWagner – I suggest holding off merging this even once approved until a validator PR is good to go as well.

Copy link
Contributor

@jskeet jskeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks Nigel.

@KalleOlaviNiemitalo
Copy link
Contributor

The wording does not seem to explain the effect of the null-forgiving operator in this case:

class C {
    static void M() {
        object obj;
        N(out obj); // warning CS8600: Converting null literal or possible null value to non-nullable type.
        N(out obj!); // no warning
    }
    
    static void N(out object? o) { o = null; }
}

: '!'
;
```

The *primary_expression* must not be known to have a value type.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this restriction relaxed after C# 8? Here's what I see at SharpLab:

class C {
    static void M() {
        (string, string?) t1 = ("", null);
        (string?, string) t2;

        t2 = t1; // warning CS8619: Nullability of reference types in value of type '(string, string?)' doesn't match target type '(string?, string)'.
        t2 = t1!; // no warning, although (string, string?) is a value type
    }

    static void N() {
        (int, int) t1 = (0, 0);
        (int, int) t2;

        t2 = t1!; // no warning, although (int, int) is a value type
    }

    static void O() {
        int t1 = 0;
        int t2;

        t2 = t1!; // no warning, although int is a value type
    }
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Roslyn started allowing e! on value types in dotnet/roslyn#32090, which is included in tag Visual-Studio-2019-Version-16.0-Preview-2. So Visual Studio 2019 version 16.0 already had this change in its C# 8 support. Visual Studio 2019 version 16.0 release notes for C#

According to dotnet/roslyn#57142 (comment) on Oct 20, 2021, "the spec" was also changed to allow ! on value types. That change was perhaps dotnet/csharplang@e8ddd37 in proposals/csharp-9.0/nullable-reference-types-specification.md, making it look like e! on value types is only allowed starting from C# 9. But AFAIK no C# 8 compiler was released without this feature.

I suppose the committee still has the authority to omit that feature from the C# 8 standard, even if compilers support it.

Comment on lines 1839 to 1842
The *primary_expression* must not be known to have a value type.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove this paragraph. I checked with the team, and it was decided that ! is allowed on a value type. The current compiler even allows it with C# 8 as the chosen version. I don't think even the C# 8 release had this change (as near as I can find in the history).

Suggested change
The *primary_expression* must not be known to have a value type.

@Nigel-Ecma Nigel-Ecma marked this pull request as draft November 1, 2024 21:38
@Nigel-Ecma
Copy link
Contributor Author

Nigel-Ecma commented Nov 1, 2024

@KalleOlaviNiemitalo & @BillWagner – I only came here to fix the grammar! ;-)

Thanks Kalle for the research and further analysis.

It's clear the original spec was never in a released compiler and so we can go straight to allowing value types.

However as Kalle has shown the implemented operator does more than override (aka “forgive”) a compiler’s static analysis determination that a null might be present – it can be a “nullability forgiving operator” and cancel any nullability warnings that a compiler might choose to provide.

I’ve moved the PR to Draft while I determine a suitable spec.

@KalleOlaviNiemitalo
Copy link
Contributor

Compiler behaviour for the null-forgiving operator in the left operand of an assignment expression looks complex:

class C
{
    void M(
        ref object o,
        ref object p)
    {
        // error CS8598
        (o!) = null;

        // warning CS8600, despite ! operator
        (o!, p!) = (1, null);

        // no warning
        (o!, p!) = this;

        // no warning
        Deconstruct(out o!, out p!);
    }
    
    void Deconstruct(
        out object? o,
        out object? p)
    {
        o = null;
        p = null;
    }
}

@KalleOlaviNiemitalo
Copy link
Contributor

KalleOlaviNiemitalo commented Nov 2, 2024

Null-forgiving ! would make perfect sense in the left operand of a compound assignment operator; but the compiler apparently doesn't allow that. The new wording should make sure that the standard doesn't allow that either.

public class C
{
    public static void M()
    {
        C? c = null;
        
        // error CS8598
        (c!) += new C();
        
        // no warning
        c = c! + new C();
    }
    
    public static C operator +(C c1, C c2)
    {
        return new C();
    }
}

@KalleOlaviNiemitalo
Copy link
Contributor

KalleOlaviNiemitalo commented Nov 3, 2024

warning CS8620 looks unjustified here; the type of the argument (p, q) should be (object p, object? q), not (object? p, object? q). It seems the compiler makes the assignment to the tuple (p, q) change the type of p to nullable even before the assignment happens. I suppose, even if it is a bug, it won't be a violation of the C# 8 standard, as compilers are allowed to output extra diagnostics.

public class C
{
    public static void M()
    {
        object p = "";
        object? q = null;
        
        // warning CS8600: Converting null literal or possible null value to non-nullable type.
        // warning CS8620: Argument of type '(object? p, object? q)' cannot be used for parameter 't' of type '(object, object?)' in '(object?, object) C.operator +((object, object?) t, C c)' due to differences in the nullability of reference types.
        (p, q) = (p, q) + new C();     
    }
    
    public static (object?, object) operator +((object, object?) t, C c)
    {
        return (null, "");
    }
}

The same happens with a named method instead of an operator:

public class C
{
    public static void M()
    {
        object p = "";
        object? q = null;
        
        // warning CS8600: Converting null literal or possible null value to non-nullable type.
        // warning CS8620: Argument of type '(object? p, object? q)' cannot be used for parameter 't' of type '(object, object?)' in '(object?, object) C.N((object, object?) t)' due to differences in the nullability of reference types.
        (p, q) = N((p, q));  
    }
    
    public static (object?, object) N((object, object?) t)
    {
        return (null, "");
    }
}

- Added null conditional support
- Distinguished uses of logical negation, `!x`, from null-forgiving, `x!`,
throughout the Standard. Null-forgiving is a non-overloadable psuedo op.
- Added placeholders in places where what the spec is has yet to be decided as the details were not obtained from the LDM notes or other design documents but from the source of one particular compiler. These will be replaced in a PR review posted immediately after this commit, each placeholder will have at least to suggested options to vote on.
- Some typos were fixed or wording changes made, some in areas not directly related to null-forgiving, as they were found during work on this PR.
Copy link
Contributor Author

@Nigel-Ecma Nigel-Ecma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In previous discussions on this feature it was known to TG2 that there was one situation where a certain compiler produced a compile time error from a use of the null-forgiving operator, rather than the usual warning.

This seemingly diverged from the design goal that the null-forgiving operator only provides information to a compiler’s optional null state static analysis to effect the analysis and its warnings.

Those discussions led to the decision to include the requirement for the error in the Standard.

Since then a certain compiler’s source code has been reviewed and a total of 5 places where that compiler produces compile time errors from usage of the null-forgiving operator found – none of which appear to be specified in any of the feature design documents.

All 5 places, including the originally known one in §12.8.9, are now marked in this PR with placeholding HTML comments.

This review provides possible text for each placeholder if the particular error is to be included in the Standard and required of all implementations, or the choice of simply removing the placeholder.

Please debate away, either on all 5 errors as a whole or on the individual placeholder comments. TG2 will review the discussion and before making final determinations.

Comment on lines +1842 to +1843
<!-- [Placeholder] 12.8.9 A -->
<!-- [Placeholder] 12.8.9 A End -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Placeholder] 12.8.9

First Alternative

A certain compiler produces a compile time error in the following situation.


It is a compile-time error to apply the null-forgiving operator more than once to the same expression, intervening parentheses notwithstanding.

Example: the following are all invalid:

var p = q!!;            // error: cannot apply the null_forgiving_operator more than once
var s = ( ( m(t) ! ) )! // error: null_forgiving_operator applied twice to m(t)

Second Alternative


Remove placeholder


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I'd prefer the first alternative. Not specifying that would limit later grammar.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BillWagner mentions the known limitation argument.

Now I might be playing the devil’s advocate here, or maybe not 😉, but why does it not apply to prefix ! or to operators such as ~?

One could also ask why it doesn’t apply to negation as well, the following crazy code is valid (please don't write it in real code!):

int m = 0;
int n = - - - --m;

which sets m to -1 and n to 1.

Given examples like this the apparent harmlessness of x!! stands out – it might earn a warning, but syntactic or semantic rules to prevent it?

Of course having the semantic rule as proposed here is also pretty harmless.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does it not apply to prefix ! or to operators such as ~?

  1. Compatibility; cannot disallow !!e after it has been allowed in earlier versions of C# (and in C, where it has been used for ensuring that an int treated as a Boolean flag is either 0 or 1).
  2. Prefix unary ! and - can be overloaded so that !!e is not the same as e (not even the same type).

Those concerns don't apply to the null-forgiving operator because it's not used in pre-existing code and cannot be overloaded.

Comment on lines +1947 to +1948
<!-- [Placeholder] 12.8.10.1 -->
<!-- [Placeholder] 12.8.10.1 End -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placeholder 12.8.10.1

First Alternative

A certain compiler produces a compile time error in the following situation.


<!-- [ToDo] C#9’s function pointers are also excluded, as the following restriction is stated in terms of what is included the text will probably be fine but this will need to be confirmed
-->
The primary_expression may be a null_forgiving_expression if and only if it has a delegate_type.


(The [ToDo] comment is part of the suggestion.)

Second Alternative


Remove placeholder


Comment on lines +2152 to +2153
<!-- [Placeholder] 12.8.11 -->
<!-- [Placeholder] 12.8.11 End -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placeholder 12.8.11

First Alternative

A certain compiler produces a compiler time error in the following situation.


<!-- [ToDo] C#9’s function pointers are also excluded, as the following restriction is stated in terms of what is included the text will probably be fine but this will need to be confirmed
-->
The optional null_forgiving_operator may be included if and only if the null_conditional_member_access or
null_conditional_element_access has a delegate_type.


(As before the [ToDo] comment is part of the suggestion.)

Second Alternative


Remove placeholder


Comment on lines +6457 to +6458
<!-- [Placeholder] 12.21.1 -->
<!-- [Placeholder] 12.21.1 End -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Placeholder] 12.21.1

First Alternative

A certain compiler produces a compile time error in the following situation.


<!-- [ToDo] This restriction also applies to null-coalescing assignment, ??= , which is a
C# 8 feature not yet spec’ed – once it is need to ensure this restriction applies.
-->
The unary_expression left operand of an assignment may not be a null_forgiving_expression.


(As before the [ToDo] comment is part of the suggestion.)

Second Alternative


Remove placeholder


Comment on lines +547 to +548
<!-- [Placeholder] 23.6.5 -->
<!-- [Placeholder] 23.6.5 End -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Placeholder] 23.6.5

First Alternative

A certain compiler produces a compile time error in the following situation.


The unary_expression operand of an address-of operator may not be a null_forgiving_expression.


Second Alternative


Remove placeholder


Comment on lines 1825 to +1827
### 12.8.9 Null-forgiving expressions

This operator sets the null state ([§8.9.5](types.md#895-nullabilities-and-null-states)) of the operand to “not null”.
A null-forgiving expression’s value, type, classification ([§12.2](expressions.md#122-expression-classifications))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because there is 12.8.9.1 too, insert a "General" heading.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about this too but left it as due to the conditionally normative bit on line 1844 (“remainder of this subclause, including all of its subclauses”). This is probably one for @RexJaeschke to wordsmith.

Comment on lines +1857 to 1860
expression cannot be null. Applying the null-forgiving operator to such an expression informs the
compiler’s static null state analysis that the null state is in *not null*; which both prevents the warning
message and informs any ongoing analysis.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"informs any ongoing analysis" apparently does not include propagating the information back to the variable from which the value originated. SharpLab:

public class C {
    public void M(int? a) {
        var b = (int)a!;
        var c = (int)a; // no warning
    }
    
    public void N(string? a) {
        var b = (string)a!;
        var c = (string)a; // warning CS8600
    }
}

In N, despite a! suggesting that a is known not to be null, the compiler issues a warning for the subsequent (string)a expression.

In M, this compiler does not warn about (int)a; however, this is apparently not caused by the null-forgiving operator in the preceding (int)a! expression, but rather by the compiler noticing that, if a were null, then (int)a! would throw and the subsequent (int)a would not be reached.

I did not check whether the specification already spells out that the null state analysis works this way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KalleOlaviNiemitalo – The specification leaves how the analysis works to the implementation, the null-forgiving operator “informs” the compiler that the user knows something but the compiler does not need to take heed… (well in e! the compiler is meant to change the determined null state, but remember that is for that specific e).

I’m not overly happy with the particular “informs any ongoing analysis” wording, and I tried and rejected other wordings, but it seems accurate – it only informs not, say, drives… Better wording suggestions welcome!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nigel-Ecma @KalleOlaviNiemitalo

Here, I think the roslyn compiler assumes certain behavior of Nullable<T>.operator T, where it throws when the instance is null.

That is by design, to avoid false positives where possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BillWagner @KalleOlaviNiemitalo – I would put it a bit more forcefully, “the Roslyn compiles knows” – and any compiler could. Whether a compiler analyses the bodies of called methods or uses the defined semantics of standard types etc. is part of the implementation specific latitude it has. Here's to the future VS vs. Rider null analysis competition! 🙂

@Nigel-Ecma Nigel-Ecma added the meeting: discuss This issue should be discussed at the next TC49-TG2 meeting label Nov 9, 2024
@@ -720,12 +720,12 @@ An *unmanaged_type* is any type that isn’t a *reference_type*, a *type_paramet

### 8.9.1 General

A *nullable reference type* is denoted by appending a `?` to a valid non-nullable reference type name. There is no semantic difference between a non-nullable reference type and its corresponding nullable type. Both a nullable reference and a non-nullable reference can contain either a reference to an object or `null`. The presence or absence of the `?` annotation declares whether an expression is intended to permit null values or not. A compiler can provide diagnostics when an expression is not used according to that intent. The null state of an expression is defined in [§8.9.5](types.md#895-nullabilities-and-null-states). An identity conversion exists among a nullable reference type and its corresponding non-nullable reference type ([§10.2.2](conversions.md#1022-identity-conversion)).
A *nullable reference type* is denoted by appending a *nullable_type_attribute* (`?`) to a non-nullable reference type. There is no semantic difference between a non-nullable reference type and its corresponding nullable type, both can either be a reference to an object or `null`. The presence or absence of the *nullable_type_attribute* declares whether an expression is intended to permit null values or not. A compiler may provide diagnostics when an expression is not used according to that intent. The null state of an expression is defined in [§8.9.5](types.md#895-nullabilities-and-null-states). An identity conversion exists among a nullable reference type and its corresponding non-nullable reference type ([§10.2.2](conversions.md#1022-identity-conversion)).

This comment was marked as resolved.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KalleOlaviNiemitalo – I forgot to add a note yesterday that nullabIe_type_attribute is introduced in PR #1178; can you move your comment there?

@Nigel-Ecma
Copy link
Contributor Author

General question: Anybody think an example explaining null! would be good and maybe also wish to suggest one, and where to included it, if so?

Copy link
Member

@BillWagner BillWagner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good. I had a few comments.

One other question for @jskeet: Typically, we've used hard breaks only for paragraphs. I don't know if the hard returns will create problems for the word converter.

@@ -787,7 +787,7 @@ When the nullable context is ***annotations***:

- For any reference type `T`, the annotation `?` in `T?` indicates that `T?` a nullable type, whereas the unannotated `T` is non-nullable.
- No diagnostic warnings related to nullability are generated.
- The null-forgiving operator `!` ([§12.8.9](expressions.md#1289-null-forgiving-expressions)) sets the null state of its operand to *not null*.
- The null-forgiving operator `!` ([§12.8.9](expressions.md#1289-null-forgiving-expressions)) may alter the analyzed null state of its operand and what compile time informative messages are produced.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've used "diagnostic messages" elsewhere to discuss the nullability analysis. I think we should be consistent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BillWagner – Thanks, I’ll go through and make sure they’re all the same.

Copy link
Contributor Author

@Nigel-Ecma Nigel-Ecma Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BillWagner – A few moments later… The Standard actually uses diagnostic to refer to both errors and warnings, as it is key the null analysis produces warnings I’ll go with diagnostic warnings (which is already used) everywhere.


> *Note*: Although `true` and `false` are not used explicitly in expressions (and therefore are not included in the precedence table in [§12.4.2](expressions.md#1242-operator-precedence-and-associativity)), they are considered operators because they are invoked in several expression contexts: Boolean expressions ([§12.24](expressions.md#1224-boolean-expressions)) and expressions involving the conditional ([§12.18](expressions.md#1218-conditional-operator)) and conditional logical operators ([§12.14](expressions.md#1214-conditional-logical-operators)). *end note*
>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will produce the format we want in the final Word output without the markdown lint warnings:

Suggested change
>
<!-- markdownlint-disable MD028 -->
<!-- markdownlint-enable MD028 -->

@@ -295,15 +295,15 @@ When overload resolution rules ([§12.6.4](expressions.md#1264-overload-resoluti

**This subclause is informative.**

Unary numeric promotion occurs for the operands of the predefined `+`, ``, and `~` unary operators. Unary numeric promotion simply consists of converting operands of type `sbyte`, `byte`, `short`, `ushort`, or `char` to type `int`. Additionally, for the unary – operator, unary numeric promotion converts operands of type `uint` to type `long`.
Unary numeric promotion occurs for the operands of the predefined `+`, `-`, and `~` unary operators. Unary numeric promotion simply consists of converting operands of type `sbyte`, `byte`, `short`, `ushort`, or `char` to type `int`. Additionally, for the unary – operator, unary numeric promotion converts operands of type `uint` to type `long`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the 2nd '-' also be the shorter character?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BillWagner – No that one is an intentional en-dash (it will have been entered directly or as two dashes and smarten will have fixed it up).

Comment on lines +1842 to +1843
<!-- [Placeholder] 12.8.9 A -->
<!-- [Placeholder] 12.8.9 A End -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I'd prefer the first alternative. Not specifying that would limit later grammar.

has the null state *maybe null* and issue a warning when other information indicates that the
expression cannot be null. Applying the null-forgiving operator to such an expression informs the
compiler’s static null state analysis that the null state is in *not null*; which both prevents the warning
message and informs any ongoing analysis.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't think of better wording, but I can think of weasel words:

Suggested change
message and informs any ongoing analysis.
message and may inform any ongoing analysis.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BillWagner – I think that is better wording and I’ll do it unless others disagree.

Comment on lines +1857 to 1860
expression cannot be null. Applying the null-forgiving operator to such an expression informs the
compiler’s static null state analysis that the null state is in *not null*; which both prevents the warning
message and informs any ongoing analysis.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nigel-Ecma @KalleOlaviNiemitalo

Here, I think the roslyn compiler assumes certain behavior of Nullable<T>.operator T, where it throws when the instance is null.

That is by design, to avoid false positives where possible.

Comment on lines +1903 to +1908
In addition to overriding *maybe null* determinations as above there may be other circumstances
where it is desired to override a compiler’s static null state analysis determination that an
expression requires one or more warnings. Applying the
null-forgiving operator to such an expression requests that the compiler
does not issue any warnings for the expression. In response a compiler may choose not
to issue warnings and may also modify its further analysis.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, this also covers ref assignment. SharpLab:

public class C {
    public void M() {
        string? s = null;
        ref string r = ref s; // warning CS8619
        r = ref s; // warning CS8619
        r = ref s!; // no warning
    }
    
    public void N() {
        (object?, dynamic?) t = (null, null);
        ref (dynamic, object) r = ref t; // warning CS8619
        r = ref t; // warning CS8619
        r = ref t!; // no warning
    }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meeting: discuss This issue should be discussed at the next TC49-TG2 meeting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants