Exported constant management process is cumbersome

[chrisnovakovic]

Since 2012, the list of libssl/libcrypto constants exported by Net::SSLeay has been managed using helper_script/regen_openssl_constants.pl. There are numerous problems with this approach:

• The script isn't well-documented. There are mistakes in the instructions for adding a constant to be exported; e.g., the suggested invocation of the script with the -gen-t option involves writing the generated test to t/local/20_autoload.t, which historically didn't exhaustively test constant loading and has since been overwritten (this presumably directly led to t/local/20_autoload.t doesn't perform any unique autoloading tests #311).
• It's written in an unclear style, making it hard to maintain (e.g., I'd like to be able to switch to a two-column format for the constants list destined for the Pod, but it's not straightforward to rework const_list_3columns to get it to do this).
• Its output is messy, and naturally leads to larger and unintelligible Git commits. For example, a three-column layout for @EXPORT_OK in lib/Net/SSLeay.pm makes it almost impossible to tell which constants have been added or removed in a commit, because every line changes. It also leads to an overall larger file size for lib/Net/SSLeay.pm than having one constant per line, because formatting each constant requires on average more than the 5 bytes it would take to indent each constant individually.
• Even with this script, multiple manual and error-prone processes need to be performed to add or remove an exported constant: multiple invocations are necessary to generate constants.c, the test file and the list of constants for lib/Net/SSLeay.pod, and the script prompts the user to manually edit @EXPORT_OK in lib/Net/SSLeay.pm (which also contains non-constant exports). This has led to inconsistencies between the four files because of the lack of clarity in the instructions; e.g., in lib/Net/SSLeay.pm the OCSP constants are listed separately from the rest of the constants for no apparent reason.

We'd benefit from a script that does for constant management what helper_script/generate-test-pki does for the PKI used by the test suite: something that consumes an exported constants list from a separate file, and automatically generates or updates all files in the repository that need to be changed as a result of that file changing. The Net::SSLeay code and documentation will be clearer, the purpose of commits will be more obvious, and there'll be fewer opportunities to make mistakes when adding or removing constants.

[h-vn]

There's also additional requirement introduced by OpenSSL 3.0.0: the return values of SSL_CTX_set_options family of functions now return an uint64_t whereas earlier they returned a long.

The help script now uses static double for constant return value. Doesn't this break things when the functions start returning values that have high bits set?

[chrisnovakovic]

There's also additional requirement introduced by OpenSSL 3.0.0: the return values of SSL_CTX_set_options family of functions now return an uint64_t whereas earlier they returned a long.

The help script now uses static double for constant return value. Doesn't this break things when the functions start returning values that have high bits set?

This is definitely something we need to think about. Since we won't be in a worse position than we already are by having a helper script that assumes all constants are doubles, I've opened another issue (#315) so we can fix this separately.