Operations, Definitions, and Classes

[webmiche]

The type[Operation] discussion hasn't really left my brain since we had it. To sketch it:

We need to use type[Operation] in our type hints sometimes. With this, we mean the actual definition of an Operation, so the stuff written in the dialect files, and not an instantiated Operation. To make this more clear from an API perspective, we had the idea of having a type alias OperationDef = type[Operation]. This is used in a lot of places throughout both the IR defintion and the IRDL stuff in order to manipulate OperationDefs. For example, a Dialect contains several OperationDefs (and AttributeDefs), a lot of attribute constraints use AttributeDefs, and so on.

IMO, this is a bit weird as it feels like we need a handle to something that normal programming languages don't want to give you a handle to, namely to the actual definition of a class. AFAIU, we are currently solving this with all the reflection-ish kinds of things in particular in the definition decorators. This is also one thing that makes typing and IDE support very hard. I am honestly not sure whether this is the best way to do it, or whether there even is another way to do it, but I would be interested in exploring this. This might also be a discussion to take over a beer in January, we will see.

To actually give us a handle to "that thing" we want, I was thinking of a factory-ish design with a structure that I will call an OperationFactory here. This will replace the Operation class as the base class to inherit from when defining new Operations. The things that are builders atm then become methods to get an instance (with the corresponding values). Honestly, I have not developed this idea further, I think that we will still have our issues around IDEs and type checking as the generated classes are still inherently dynamic, which makes type checking hard. However, for me this interface looks more understandable and it separates the instantiated operation classes more from the operation definitions and gives us a handle to the latter. This might even be a fundamental advantage for type checking. In my wildest dreams, I could imagine somehow "generating the python class" for an Operation (in actual python code) and handing it to the IDE to allow auto completion and stuff.

[math-fehr]

So multiple thoughts here.

First of all, about type checking, I do not think that this is inherently a problem of using type[Operation], since I think that OperationDef would not fix this part. Generating files could be interesting, however I'm a bit scared of doing this, because it changes drastically the pipeline (i.e., we would need to generate files, so we loose the Python dynamicity). However, I would be happy to generate files for some use cases (like the MLIR dialects).

Also, I would disagree that "normal" languages do not allow you to write type[Operation]. In C++ for instance, you would just write template <Op>, which correspond to the exact same thing. The difference though, is that Python allows you to write that is a simple operand of a function. Looking at it this way, I still thing that type[Operation] represent what we want, which is exactly an OperationFactory, but with actually better types ;)

[webmiche]

So you are saying that the current approach already does that somewhat, but with allowing typing to work as well?

I see what you mean. Maybe I am just not yet used to meta programming and reflection-ish stuff in a way that makes me very comfortable 😅

I can see what you mean. Maybe an option like generate files when running xDSL would make sense, which would generate (and update) some cache class files that allow introspection of types for the IDE?

[math-fehr]

And yes, I agree that we should discuss that further around a drink in January!