[crypto] Trial division for RSA key generation.

[jadephilipoom]

Filter out 62.1% of composite numbers by using arithmetic tricks specific to certain small primes. This should speed up RSA key generation by something close to 60% in expectation, because primality testing is the vast majority of the runtime and this lets us skip it in 60% of cases where it will eventually fail.

[rswarbrick]

I've just had a really enjoyable few minutes reading through some of this code! (I must say, I hadn't thought of using a ones-register to do masking like this: it's neat!)

A couple of nitty comments on the bits that I'd read.

[rswarbrick]

I think this might need tweaking to match the "& 8" that's now in the code?

Looking at the rest of the file, it looks like there are several other places where we could make the corresponding change. Maybe that would be worth doing uniformly as a separate commit?

[jadephilipoom]

Maybe so, yes. I think I was looking to save code size here.

[rswarbrick]

Maybe "at most" instead of "<=", since those symbols are also being used for assignment?

[rswarbrick]

"Testing "??

(I think the rest of the paragraph appears in the next commit?)

[jadephilipoom]

Not sure what was going on there! Anyway, I double-checked and it does seem like it's filled in on the next commit. The full paragraph is:

 * Testing for these primes will catch approximately:
 *   1 - ((1 - 1/3) * (1 - 1/5) * ... * (1 - 1/31))
 *   = 62.1% of composite numbers.

Maybe I just hadn't done the calculation yet 🙂

[jadephilipoom]

I was searching for my own open PRs recently and noticed this one was never merged! Looks like it was approved the day before I changed employers and lost my open tabs so I think I know how that happened 😉 Luckily, the code hasn't changed much in the meantime, so I've rebased it and unless there are objections I'll merge once it passes CI.

[jadephilipoom]

Since we now have nicer profiling targets than we did when I first made this PR, I also ran some quick benchmarks. I modified the RSA-2048 keygen test to generate keys 10 times and continue recording the OTBN instruction count (a good proxy for the cycle count, and more accurate on FPGA tests given RSA keygen is too slow for Verilator). The results were:

>>> master = [728920834,626858263,844018075,212064556,1295722633,1265332333,1395617782,798422326,1964599999,225096775]
>>> trial_div = [518505838,279511222,642358674,344699495,1109487354,1467981868,223009163,527194053,118732085,535880793]
>>> avg(master)
935665357.6
>>> avg(trial_div)
576736054.5
>>> avg(master) / avg(trial_div)
1.6223458726040163

In short, I observed a 62% speedup, a nice confirmation of the 61% estimate. I think I just got lucky with how exact it is; the standard deviation on these measurements is large because RSA timing is so variable and I only had the patience for 10 samples. But still -- at least it confirms that we see the significant average speedups we expect.