PyNumber_InPlacePower ignores o3 if o1 implements __ipow__

[hodgestar]

BPO	41903
Nosy	@brettcannon, @ammaraskar

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-01.15:10:02.476>
labels = ['type-bug', '3.8', '3.9', '3.10', 'expert-C-API', '3.7']
title = 'PyNumber_InPlacePower ignores o3 if o1 implements __ipow__'
updated_at = <Date 2020-10-01.15:48:52.092>
user = 'https://bugs.python.org/hodgestar'

bugs.python.org fields:

activity = <Date 2020-10-01.15:48:52.092>
actor = 'ammar2'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2020-10-01.15:10:02.476>
creator = 'hodgestar'
dependencies = []
files = []
hgrepos = []
issue_num = 41903
keywords = []
message_count = 5.0
messages = ['377758', '377760', '377761', '377766', '377767']
nosy_count = 3.0
nosy_names = ['brett.cannon', 'hodgestar', 'ammar2']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41903'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

• gh-86069: correct object.__ipow__() signature #130103
• gh-86069: Add more PyNumber_InPlacePower() tests #130111
• [3.13] gh-86069: Add more PyNumber_InPlacePower() tests (GH-130111) #130211
• [3.12] gh-86069: Add more PyNumber_InPlacePower() tests (GH-130111) #130212
• gh-86069: Do not ignore the third argument in slot_nb_inplace_power() #130225

[hodgestar]

The documentation for PyNumber_InPlacePower [1] reads:

This is the equivalent of the Python statement o1 **= o2 when o3 is Py_None, or an in-place variant of pow(o1, o2, o3) otherwise. If o3 is to be ignored, pass Py_None in its place (passing NULL for o3 would cause an illegal memory access).

However, if a class A implements __ipow__ then PyNumber_InPlacePower(o1, o2, o3) ALWAYS ignores o3 if o1 is an instance of A.

This happens because if A implements __ipow__ then PyNumber_InPlacePower always calls the nb_inplace_power slot [2] and the slot always drops the third argument [3].

This appears to have always been the case in Python, so likely a small documentation patch is all that is required. If people agree, I will open a documentation pull request.

[1] https://docs.python.org/3/c-api/number.html?highlight=pynumber_inplacepower#c.PyNumber_InPlacePower

[2] https://github.com/python/cpython/blob/master/Objects/abstract.c#L1164

[3] https://github.com/python/cpython/blob/master/Objects/typeobject.c#L6631-L6636

[hodgestar]

The documentation for __ipow__ [4] suggests that the intention was to support the modulus argument, so perhaps that argues for fixing the behaviour of PyNumber_InPlacePower.

[4] https://docs.python.org/3/reference/datamodel.html#object.\_\_ipow__

[ammaraskar]

Huh, this is weird. I wonder why the data model provides the signature

  object.__ipow__(self, other[, modulo])

if the slot always seems to drop the third argument.

Adding Brett to the nosy who found another issue around the special casing of pow/** recently. See also bpo-38302

[ammaraskar]

Also of note, there is actually no way to call PyNumber_InPlacePower from pure python from what I can tell. Even libraries like numpy don't implement the three-argument version of the nb_inplace_power slot.

[ammaraskar]

Whoops, forgot the numpy link: https://github.com/numpy/numpy/blob/33e1dbee8d9e11a3e96efaae822ff6f3c44e3cef/numpy/core/src/multiarray/number.c#L729-L741

[skirpichev]

This looks like a documentation issue for me.

CC @picnixz
CC @serhiy-storchaka

[serhiy-storchaka]

The default implementation of the nb_inplace_power slot, slot_nb_inplace_power() in Objects/typeobject.c, ignores the third argument and calls the __ipow__() method with only two arguments (including self). But an extension class can implement the nb_inplace_power slot that does not ignore the third argument. The __ipow__() wrapper accepts 2 or 3 arguments (including self), all of them are passed to the nb_inplace_power slot, which can not ignore the third argument.

This looks like a design mistake. PyNumber_InPlacePower() should not have the third argument, and __ipow__() should not accept the third argument. But currently they do, and we are stuck with this.

The fact that the third argument of PyNumber_InPlacePower() usually is not passed to the __ipow__() method cab be considered a bug. We can fix this or we can document this wart, although it is not easy to document it without referring to the implementation details (and not easy to describe the change in the behavior if we fix it).

[skirpichev]

This looks like a design mistake.

Lets correct this! There is no way to call the 3-arg dunder with some Python API (like pow()), only directly. Or even with the current C-API (PyNumber_InPlacePower).

usually is not passed

Never. At least in the current implementation.

[serhiy-storchaka]

The current behavior is inconsistent. If __ipow__ is not defined, then the third argument is not ignored, but passed to __pow__. If __ipow__ is defined, then the third argument is ignored. For extension classes it is not ignored, it is only ignored for Python classes.

One way to solve this is to stop ignoring the third argument for __ipow__. #130225 implements this way.

Other way -- deprecate passing non-None third argument in PyNumber_InPlacePower() and in the __ipow__ wrapper.