Merge #593

593: Support RangeFrom ([x..]) and RangeFromTo ([x..y]) in the parser r=philberty a=dkm

Parsing the .. (DOT_DOT) operator to get a range had two
issues. Trying to compile:

  let block = [1,2,3,4,5];
  let _rf = &block[1..];
  let _rt = &block[..3];
  let _rft = &block[2..4];

range.rs:4:23: error: found unexpected token ‘]’ in null denotation
    4 |   let _rf = &block[1..];
      |                       ^
range.rs:4:24: error: expecting ‘]’ but ‘;’ found
    4 |   let _rf = &block[1..];
      |                        ^

Since .. can represent either a range from or a range from-to it can
be followed by an expression or not. We do have a hack in our
pratt-parser so that it is allowed to return a nullptr. But even in
that case it will have swallowed the next token. Add another hack to
the pratt-parser so that if the next token is one that cannot start an
expression and the caller allows a nullptr return then don't skip the
token and return immediately.

After this patch we can parse the above range expressions, but we
still don't handle them fully:

range.rs:4:20: fatal error: Failed to lower expr: [1..]
    4 |   let _rf = &block[1..];
      |                    ^

Ranges are actually syntactic sugar for std::ops::Range[From|To].

Co-authored-by: Mark Wielaard <[email protected]>

Author: bors[bot]

gcc/rust/parse/rust-parse-impl.h

@@ -12348,6 +12348,18 @@ Parser<ManagedTokenSource>::parse_expr (int right_binding_power,
 					ParseRestrictions restrictions)
 {
   const_TokenPtr current_token = lexer.peek_token ();
+  // Special hack because we are allowed to return nullptr, in that case we
+  // don't want to skip the token, since we don't actually parse it. But if
+  // null isn't allowed it indicates an error, and we want to skip past that.
+  // So return early if it is one of the tokens that ends an expression
+  // (or at least cannot start a new expression).
+  if (restrictions.expr_can_be_null)
+    {
+      TokenId id = current_token->get_id ();
+      if (id == SEMICOLON || id == RIGHT_PAREN || id == RIGHT_CURLY
+	  || id == RIGHT_SQUARE)
+	return nullptr;
+    }
   lexer.skip_token ();
 
   // parse null denotation (unary part of expression)
@@ -14036,6 +14048,9 @@ Parser<ManagedTokenSource>::parse_led_range_exclusive_expr (
 {
   // FIXME: this probably parses expressions accidently or whatever
   // try parsing RHS (as tok has already been consumed in parse_expression)
+  // Can be nullptr, in which case it is a RangeFromExpr, otherwise a
+  // RangeFromToExpr.
+  restrictions.expr_can_be_null = true;
   std::unique_ptr<AST::Expr> right
     = parse_expr (LBP_DOT_DOT, AST::AttrVec (), restrictions);