[WIP] Expansion of #[derive(Trait)] macro attributes

[eupn]

In this WIP PR, I'll try to build a ground to further resolve #1320 and #1427 by adding more thorough parsing and collecting of #[derive(...)], #[must_use] and other attributes and embedding them into code model or HIR so we can operate on them more easily. This would enable diagnostics and code generation for derives (as suggested by @matklad in #1427 (comment)).

For now, I publishing this mostly to collect some feedback regarding design decisions and mentoring.

• Collecting of #[attr] attributes as macros
• Expansion of #[derive(Trait)] <item> macros into impl Trait for <item> {}
  • Expansion of #[derive(Trait)] <item> for items with generic types
  • Error reporting on empty trait list: #[derive] should produce an error message (?)

@matklad, @flodiebold

[matklad]

It's better if this renmains `pub(crate)

[eupn]

Produces the following:

error[E0446]: crate-visible type `source_id::AstId<ra_syntax::ast::generated::MacroCall>` in public interface
   --> crates/ra_hir/src/ids.rs:215:30
    |
215 |     Macro { def: MacroDefId, ast_id: AstId<ast::MacroCall> },
    |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't leak crate-visible type
    | 
   ::: crates/ra_hir/src/source_id.rs:19:1
    |
19  | pub(crate) struct AstId<N: AstNode> {
    | ---------- `source_id::AstId<ra_syntax::ast::generated::MacroCall>` declared as crate-visible

[matklad]

I think if we store attributes as token trees in raw times, we wouldn't need to introduce ids for attributes, and this seems like a good thing to do

[eupn]

Not sure what to do to resolve this.

[flodiebold]

this won't work for generic types, but we can fix that later

[eupn]

@flodiebold @matklad I've implemented an inference test, but it doesn't seem to work. The type inferred is {unknown}. The problem is that #[derive(Clone)] expands into impl Clone for S {} token tree, but that doesn't seem to be correctly reflected further in HIR. If I replace derive with impl block, everything works fine. Also, if I produce an impl block with macro_rules! macro, the result is {unknown} too.

[eupn]

I think the problem is that macros are expanding into different file or module. Or some plumbing is still missing.

[flodiebold]

Ah, good to know... My guess would be that it's because impl blocks are collected separate from name resolution, though I don't know the macro infrastructure so well so I might be wrong. I'm not sure what the proper way to fix that would be.

[flodiebold]

@matklad What do you think we should do here? Currently impl blocks are collected completely separately from nameres, but to keep that up we'd have to do macro expansion again there, I guess... Should DefCollector just collect impl blocks as well?

[eupn]

Yeah, I kinda stuck with this issue. Should try to collect impl blocks too somehow.

[matklad]

impl block collection was imled in #1861

[eupn]

Now, this test passes! 🎉

[matklad]

I have a feeling that this PR is getting out of hand, being sitting for more than a month and accumulating 25+ commits. However, I don't feel like the current state is what would be good to support long term. I see at least two problems:

• there are some changes to ra_syntax layer, which shouldn't be there, because syntactically attributes are just stuff between #[...] and this we fully support already. I feel like any changes to the syntax here will inevitably mix in semantics with syntax, and I'd love to avoid that as much as poissible

• the actual expansion works on the ast, and not on the token trees. This is problemantic, because it makes incremental hard (as syntax trees change with every file change)

Now, the best way to solve this would be for me to write "ten easy steps to implement attribute macro expansion in rust-analyzer", but unfortunately I don't have a precise design in mind. I also don't have time, right now, to produce such a design. I suggest putting the actual hacking on pause for a moment and try to co-design the end state of attribute expansion here:

https://hackmd.io/@Ngg7W1yWSMySlEhnj8E5sw/HyyqCzIPB

I am not sure that this is the right approach to designing things, but I definitely thing that repeatedly commenting on a PR which doesn't get the high-level structure right is not a good approach either.