Python 3.10: TypeError: __add__() got an unexpected keyword argument 'context'

[rndstr]

Running the example i'm getting this error (tested with Python3.9 and Python3.10)

 $ rp2_us -m fifo -o output -p crypto_example_ crypto_example.config crypto_example.ods
INFO: Country: us
INFO: Accounting Method: fifo
INFO: Configuration file: crypto_example.config
INFO: Input file: crypto_example.ods
INFO: Processing BTC
ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/rp2_main.py", line 93, in _rp2_main_internal
    input_data: InputData = parse_ods(configuration=configuration, asset=asset, input_file_handle=input_file_handle)
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/ods_parser.py", line 128, in parse_ods
    _create_and_process_transaction(
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/ods_parser.py", line 168, in _create_and_process_transaction
    transaction: AbstractTransaction = _create_transaction(configuration, current_table_type, internal_id, row_values)
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/ods_parser.py", line 277, in _create_transaction
    transaction = InTransaction(**argument_pack)
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/in_transaction.py", line 52, in __init__
    super().__init__(configuration, timestamp, asset, transaction_type, spot_price, internal_id, unique_id, notes)
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/abstract_transaction.py", line 41, in __init__
    self.__spot_price: RP2Decimal = configuration.type_check_positive_decimal("spot_price", spot_price)
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/configuration.py", line 276, in type_check_positive_decimal
    if result < ZERO:
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/rp2_decimal.py", line 59, in __lt__
    return not self.__ge__(other)
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/rp2_decimal.py", line 48, in __ge__
    return (self - other).quantize(CRYPTO_DECIMAL_MASK).__ge__(ZERO)
  File "/home/rndstr/venv/lib/python3.9/site-packages/rp2/rp2_decimal.py", line 69, in __sub__
    return RP2Decimal(Decimal.__sub__(self, other))
  File "/usr/lib/python3.9/_pydecimal.py", line 1257, in __sub__
    return self.__add__(other.copy_negate(), context=context)
TypeError: __add__() got an unexpected keyword argument 'context'
INFO: Log file: ./log/rp2_2022_04_10_15_52_55_045185.log
INFO: Generated output directory: output
INFO: Done

[eprbell]

Which version of RP2 are you using? If not the latest, could you upgrade and try again? Also make sure the example ODS and config are the latest version.

[rndstr]

I set everything up yesterday and was using 0.9.29 installed with pip.

[rndstr]

It looks like you are overriding Decimal.__add__() but do not provide the context param which is in the Decimal class in Python3.9/3.10:

https://github.com/python/cpython/blob/89697f7374ea947ebe8e36131e2d3e21fff6fa1d/Lib/_pydecimal.py#L1157

and then being called virtually

https://github.com/python/cpython/blob/f33e2c87a83917b5139d97fd8ef7cba7223ebef5/Lib/_pydecimal.py#L1257 (this goes back until at least 3.7)

not sure why it would work for others--if that's the case--though.

[eprbell]

Interesting: haven't seen that before. The crypto_example.ods test is ran as part of continuous integration at every checkin on Python 3.7, 3.8, 3.9 and on Linux, Windows and Mac. A few more questions:

• what OS and OS version are you using?
• did you install Python as described in the docs or did you use a Python that came with the OS?

[rndstr]

Can confirm elevating that signature of RP2Decimal.py fixes it, (PR at #28)

ArchLinux, Linux hostname 5.17.1-arch1-1 #1 SMP PREEMPT Mon, 28 Mar 2022 20:55:33 +0000 x86_64 GNU/Linux

I used a clean virtualenv with Python3.10 that came with the OS.

[eprbell]

The unit tests are failing in the PR:

Traceback (most recent call last):
  File "/home/runner/work/rp2/rp2/src/rp2/rp2_main.py", line 93, in _rp2_main_internal
    input_data: InputData = parse_ods(configuration=configuration, asset=asset, input_file_handle=input_file_handle)
  File "/home/runner/work/rp2/rp2/src/rp2/ods_parser.py", line 128, in parse_ods
    _create_and_process_transaction(
  File "/home/runner/work/rp2/rp2/src/rp2/ods_parser.py", line 168, in _create_and_process_transaction
    transaction: AbstractTransaction = _create_transaction(configuration, current_table_type, internal_id, row_values)
  File "/home/runner/work/rp2/rp2/src/rp2/ods_parser.py", line 277, in _create_transaction
    transaction = InTransaction(**argument_pack)
  File "/home/runner/work/rp2/rp2/src/rp2/in_transaction.py", line 81, in __init__
    self.__fiat_in_with_fee = self.__fiat_in_no_fee + self.__fiat_fee
  File "/home/runner/work/rp2/rp2/src/rp2/rp2_decimal.py", line 64, in __add__
    return RP2Decimal(Decimal.__add__(self, other, context))
TypeError: expected 1 argument, got 2

The context is probably something that got added in Python 3.10 (unit tests are running on 3.7, 3.8 and 3.9).

[rndstr]

Odd, I see it in 3.7 as well https://github.com/python/cpython/blob/3.7/Lib/_pydecimal.py#L1157

[rndstr]

Is there some sort of non-strict mode in Python that would allow extra arguments in this particular instance even though it's not part of the signature?

[eprbell]

Not sure... but it looks like RP2 is not working yet on Python 3.10. If you find a way to make the PR work in a backward compatible way, I'll accept it. For now the (not ideal) workaround is to use Python 3.9.

[eprbell]

I tried with Python 3.10 on my personal machine and I couldn't repro this problem. I also added Python 3.10 to continuous integration on Windows, Mac and Linux and it all works well (see 0d7b01a). I think the issue you are seeing might be due to some specific problem with your Python installation.

[eprbell]

Here is continuous integration running on Python 3.10:

• Windows: https://github.com/eprbell/rp2/runs/5993615909?check_suite_focus=true
• Mac: https://github.com/eprbell/rp2/runs/5993616027?check_suite_focus=true
• Linux: https://github.com/eprbell/rp2/runs/5993615670?check_suite_focus=true

You may want to try uninstalling python and reinstalling it.

[eprbell]

I don't think there is anything else actionable on this issue. OK to close?

[antonok-edm]

FWIW, I think it has to due with the fact that there are two possible backends for the decimal module, see cpython/Lib/decimal.py.

I confirmed that on Arch Linux it's loading from _pydecimal. I assume that elsewhere it's loading from _decimal instead.

[antonok-edm]

Found it!

> python
Python 3.11.8 (main, Feb 12 2024, 14:50:05) [GCC 13.2.1 20230801] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _decimal import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: libmpdec.so.4: cannot open shared object file: No such file or directory

> pacman -Ss mpdec
extra/mpdecimal 4.0.0-2
    Package for correctly-rounded arbitrary precision decimal floating point
    arithmetic
> sudo pacman -Syu mpdecimal
...
> python
Python 3.11.8 (main, Feb 12 2024, 14:50:05) [GCC 13.2.1 20230801] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _decimal import *
>>>

...and rp2 appears to be working after this process as well.

TLDR: mpdecimal is a required dependency of rp2. I guess on Ubuntu it's either installed by default with the OS or with the distribution of Python. On Arch it must be installed separately.

[eprbell]

Fantastic! Thanks for the good detective work :-)

[antonok-edm]

I filed a bug against Python; maybe this could be fixed upstream.

[eprbell]

Good idea. If you also want to submit a PR to add the dependency for RP2 I'll approve it.

[eprbell]

This should now be resolved (see #108).