AVM: Adding bmodexp

[mangoplane]

Summary

Adds the new opcode bmodexp as described in issue #6139 to support modular exponentiation involving byte strings of up to 4096 bytes. Closes #6139

Test Plan

• Relevant tests added to assembler_test.go, evalStateful_test.go, & eval_test.go
• Opcode is tested with a range of test vectors with function TestBytesModExp, covering panic cases, edge cases, acceptance cases and failure cases. Test vectors were generated manually or with Python.

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 56.23%. Comparing base (2a02530) to head (7b6b6c2).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6140      +/-   ##
==========================================
- Coverage   56.27%   56.23%   -0.04%     
==========================================
  Files         494      494              
  Lines       69943    69957      +14     
==========================================
- Hits        39358    39341      -17     
- Misses      27910    27932      +22     
- Partials     2675     2684       +9     

Flag	Coverage Δ
	56.23% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jannotti]

This is looking quite good so far.

[jannotti]

I'm doing this from phone, but maxStringSize is the AVM max of 4096? That's unusual for the bmath functions, they normally have a maximum of 128 bytes. I suppose you want more because RSA uses really large keys?

At any rate, it's impossible to supply inputs that are larger than this, so there's no need to check in the opcode.

[mangoplane]

That makes sense. Yes, the size is intended to support RSA which has really large keys. Is it okay if we allow the opcode to support this size?

[jannotti]

It's ok with me, assuming we can get the cost function to properly account for long inputs. It seems like bmodexp wouldn't be very interesting if it can't handle common RSA key sizes.

I'd like to support bigger inputs on the other b opcodes too, if we can adjust cost functions appropriately. They were first done before we could have the cost depend on size. b+, for example, would almost certainly be easy to adjust, b* might be more complex than simple length dependence. I'm somewhat worried that bmodexp is going to be tricky. Anyway, that should be a separate PR someday. (Do your RSA implementations require operating on RSA keys with any other operations?)

[jannotti]

If it ends up being fast enough, we could pick a cost that accounts for the worst case, which I suppose would three very long inputs.

Or perhaps only scales based on one of the parameters, but accounts for the worst case with the others. (I think bmodexp should be linear with respect to the length of the exponent, since it basically performs one operation per bit.)

[jannotti]

I just read your discussion on cost from the issue more closely. The dependence on the square of the lengths is unfortunate because it implies a custom cost function. Currently, all costs are "data directed" which is nice because it means we can create specs automatically - we can generate text that describes the cost from the constants provided in the opcode table.

I suppose we can add a way to provide both a Go function and a textual description directly in the .go source code. That is somewhat more fragile from the standpoint of modifications to the way we present the spec, but it doesn't see so bad. It's probably also necessary if we ever want to support larger inputs to b* which I suspect is where this really coming from.

[mangoplane]

If it ends up being fast enough, we could pick a cost that accounts for the worst case, which I suppose would three very long inputs.

Or perhaps only scales based on one of the parameters, but accounts for the worst case with the others. (I think bmodexp should be linear with respect to the length of the exponent, since it basically performs one operation per bit.)

This is very tempting, and would mean minimal complexity. I anticipate bmodexp rarely being used, except for applications where the inputs are large such as RSA.

Or perhaps only scales based on one of the parameters, but accounts for the worst case with the others. (I think bmodexp should be linear with respect to the length of the exponent, since it basically performs one operation per bit.)

This is a good idea that might simplify the calculation to allow an existing linear cost model to be used.

Do your RSA implementations require operating on RSA keys with any other operations?

The only operators for RSA besides modexp are the less than and equal operators. These are efficiently implemented with a 512 bit digit partitioning algorithm available in Puya-Bignumber.

I will explore the suggestions to try to linearise the cost model, and figure out the max cost to see if it's cheap enough that we can use a constant. In my opinion, I think the complexity that would be introduced as highlighted with a non-linear cost model isn't worth it, if we can figure out a bounding linear model that's good enough. I'll present the results of the linear model to the discussion thread, and go from there.

[jannotti]

Was this required to make the test pass? I wouldn't have expected it to be. This test is really only a sanity check on typing, so if you did this just to exercise the opcode more, I'd rather you do that all in the opcode specific tests.

[mangoplane]

Was this required to make the test pass?

It's intended as another sanity check following convention, although I don't know if it's required.

[jannotti]

The convention is to only add an element to that table if it's required. For the majority of opcodes, the test conjures up "reasonable" inputs (like a number for integer inputs or a short string for byte inputs). This table is for opcodes that need specific inputs in order to function.

[jannotti]

I think we can dispense with the elaborate version constants and "experimental" stuff for this opcode. When we merge it, we're committing to get it into the next version.

[mangoplane]

I have addressed your feedback with a recent update, per my understanding. Let me know if there's anything that I may have misinterpreted.

[giuliop]

bmodexp is useful also in different scenarios with smaller inputs so we should not penalize those cost-wise, for instance all ZKP protocols based on elliptic curves use it and they try to operate on the smallest field possible while preserving security for efficiency.

As a concrete example, smart contract verifiers for zk-proofs based on the plonk protocol generated by AlgoPlonk call a teal implementation of it 5 times, using 32-byte inputs for both curve BN254 and BLS12-381.

Considering that a BN254 verifier consumes ~145,000 opcode budget and a BLS12-381 verifier ~185,000, bmodexp would really help bring down that cost.

[giuliop]

I'm doing this from phone, but maxStringSize is the AVM max of 4096? That's unusual for the bmath functions, they normally have a maximum of 128 bytes. I suppose you want more because RSA uses really large keys?

Isn't the maximum 64 bytes?

If we can have a plausible linear model for the smaller inputs currently supported by the b-operations (64 bytes?) which breaks for very large inputs, a solution we might consider for consistency is to offer bmodexp that operates on bigint like the other b-operations and add a separate fixed-cost opcode that operates on 4096 byte strings, e.g. modexpstring

[mangoplane]

I'm doing this from phone, but maxStringSize is the AVM max of 4096? That's unusual for the bmath functions, they normally have a maximum of 128 bytes. I suppose you want more because RSA uses really large keys?

Isn't the maximum 64 bytes?

If we can have a plausible linear model for the smaller inputs currently supported by the b-operations (64 bytes?) which breaks for very large inputs, a solution we might consider for consistency is to offer bmodexp that operates on bigint like the other b-operations and add a separate fixed-cost opcode that operates on 4096 byte strings, e.g. modexpstring

You make some good points. However, in my opinion, we should offer a single opcode to reduce complexity. With a sufficiently accurate cost model, such as the one proposed in this GitHub comment, the estimated cost will closely reflect the actual cost within a small margin of error. For example, using the log-polynomial model, the cost for 64 byte long input is 105, which is intuitively reasonable and relatively small. To simplify the cost in that range, we could use a piecewise function where the cost is 105 for all inputs up to 64 bytes in length, and for longer inputs, it follows the advanced cost model. The log-polynomial model also accurately describes the cost for much larger inputs.

Additionally, this seems to align with the long-term vision of allowing byte math opcodes, such as b+, to support up to 4096 bytes each, if I'm not mistaken based on the above discussion.

The opcode should work for inputs up to 4096 bytes, with several test cases exceeding the 64-byte limit. As a sanity check I think it's worth adding another test case to verify the maximum supported length of 4096 bytes.