New InvalidCallError or InvalidArgsError subtype of TypeError to favour duck-typing compliant callbacks usage pattern ; AND/OR more robust signature inspection tools

[smarie]

It is frequent that libraries wish to receive user-provided callbacks. For example consider the following function:

def register_callback(user_provided_callback):
    ...

A use case that I met several times in my libraries is to accept both simple (one argument) and complex (more arguments) callbacks. A good example is pyfields converters, stating:

The conversion function should be a callable with signature f(value), f(obj/self, value), or f(obj/self, field, value), returning the converted value in case of success and raising an exception in case of conversion failure.

As of today it appears that the only way to handle this is to inspect the signatures. This is what I do for example in pyfields' make_3params_callable. It can not rely on getfullargspec since getfullargspec does not have a skip_bound_arg argument (this was proposed but rejected in this discussion) ; it needs to rely on inspect.signature / inspect.Signature.from_callable.

I see two major points of concern here

• first unfortunately inspect.signature is not guaranteed to return a proper result and not to raise an error if you provide a valid callable. I came across many examples of callables that make inspect.signature raise an error, most of them beeing built-in functions or classes (constructors). For example signature(str) raises a ValueError, while the constructor of the str class is a well-defined callable. Handling all these edge cases is annoying and leads to ridiculously complex implementations (my getfullargspec is a good example)

• second, implementing this in a duck-typing style is by design not possible. Indeed the error returned by the python interpreter when the call is invalid (i.e. if I call the callback with 3 arguments but it only accepts 2) is a plain old TypeError ; I have no possibility to distinguish it from a TypeError that would be raised by the callback itself as part of normal usage.

My proposals for a PEP to solve this issue:

• First, create a new exception type for example named InvalidArgSpecError, InvalidArgsError, or InvalidCallError. That would be a subtype of TypeError, that would be raised when the number of arguments provided is not correct. For example it would be raised when you do (lambda a, b: 1)(2). Currently you receive a TypeError: <lambda>() missing 1 required positional argument: 'b'. Tomorrow if this proposal is accepted you would receive an InvalidCallError: <lambda>() missing 1 required positional argument: 'b'. Note that we could even imagine several error subtypes of InvalidCallError to distinguish between MissingMandatoryArgumentError, InvalidPositionalOnlyArgument, InvalidKeywordOnlyArgument, etc. This proposal will allow us to quickly implement duck-typing callable usage:

# call the user-provided callback callable according to the number of arguments it supports
try:
    user_provided_callback(a, b, c)  # 3-args callback
except InvalidCallError:
    try:
        user_provided_callback(a, b)  # 2-args callback
    except InvalidCallError:
        user_provided_callback(a)  # 1-arg callback

• Second, enforce the requirement for callable-inspecting methods such as signature or getfullargspec, to never raise an error if the inspected object is a valid callable (partial, functions, built-ins, partials of built-ins, instance, class or metaclass methods, bound or unbound). If this is not possible by design, specify a new function in the stdlib that would have more limited inspection capabilities, but would be guaranteed to support all the above possible callables. After all, in the use case that I describe above, only the number of parameters and their kind is required - not their names.