Skip to content

Latest commit

 

History

History
66 lines (51 loc) · 2.83 KB

README.md

File metadata and controls

66 lines (51 loc) · 2.83 KB
Raw

Validation

Dart repointer

Dart repointer (reptr) puts the sharp part in your Dart aims to facilitate fast code generation for Dart. It consists of the tool itself and a Dart parser library.

Motivation

build_runner-based code generation for a single package can take a couple minutes. This translates to unreasonably long code generation time for large Dart and Flutter projects (e.g. over 45 minutes 🤯). This is predominantly due to inefficiencies in the code generation infrastructure, the build_runner. A faster implementation should allow 20–50x speed-up.

Project structure

The key components of this project that are already functional are:

  • Filesystem scanner
    • Identifies Dart packages and source files on the filesystem
  • File loader
    • Loads a file into memory, performs UTF-8 validation
  • Parser (lexerless)
    • Constructs an abstract syntax tree from the loaded Dart code
flowchart LR
  CLI --> Scanner[F/s scanner] --> Loader --> Parser["Parser\n(lexerless)"] --> Gen1 & Gen2 --> Combiner --> Writer --> Formatter[External formatter]

  CLI --> Reporter[Status / progress reporter]

  CLI --> Watcher[F/s watcher]

  subgraph Generator 1
    Gen1[...]
  end

  subgraph Generator 2
    Gen2[...]
  end

  style Reporter stroke-dasharray: 5 5
  style Watcher stroke-dasharray: 5 5
  style Gen1 stroke-dasharray: 5 5
  style Gen2 stroke-dasharray: 5 5
  style Combiner stroke-dasharray: 5 5
  style Writer stroke-dasharray: 5 5
  style Formatter stroke-dasharray: 5 5
Loading

Design notes

  • Loading source files
    • Loading the complete file for parsing is fine
    • Memory-mapping is not faster per se
      • May play nicely with a lazy tokenizer with lazy UTF-8 validation
  • Parsing
    • The parser is a partial parser: it recognizes certain parts of the target language and can skip over the rest
    • Capturing slices of the source (&str) is extremely cheap (doesn't cause memory allocation), so take advantage of that
    • When feasible, avoid memory allocation (namely, the use of Vec and co.)
    • When implementing a parser as a function
      • Do not start parsing with whitespace
        • E.g. the import-stmt. parser should start with consuming import, not whitespace
      • Do not consume the trailing whitespace after a construct (e.g. in import 'dart:math';\n\n do not consume \n\n)
    • When combining parsers
      • Prefer consuming whitespace in trailing position, not leading
  • A tempting feature: in-place code generation / code transformation
    • Requires accurate back-to-source transformation
  • Output formatting