rsa: bugfix about redundant and inefficient operations

Includes:
1. Remove redundant judgment conditions.
2. Remove redundant function parameters.
3. Remove the redundant operation in soft RSA keygen method.
4. Use more efficient BN memory allocation method, and add
judgment of the results.

Signed-off-by: Zhiqi Song <[email protected]>