Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement High-Quality Random Number Generation Using AES-CTR Mode with OpenSSL and AES-NI Support #615

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Knogle
Copy link
Contributor

@Knogle Knogle commented Oct 9, 2024

Implement Cryptographically Secure PRNG Using AES-256-CTR with OpenSSL and AES-NI Support

This commit introduces a new cryptographically secure pseudorandom number generator (CSPRNG) for Nwipe, utilizing the AES-256 algorithm in Counter mode (CTR). By integrating with OpenSSL and leveraging AES-NI hardware acceleration when available, this implementation provides high-quality random numbers essential for cryptographic operations and secure data wiping.

Algorithms have been added in order to ensure high PRNG quality in case of some issue with the OpenSSL library.
4K of random data will be generated during init with the actual seed, and checked for randomness and entropy.

Furthermore i suggest closing #604 in favor of this one, as there is no benefit if we implement AES.

int nwipe_aes_ctr_prng_init(NWIPE_PRNG_INIT_SIGNATURE)
{
    /* rest of the code here */

    // Validate the PRNG output to ensure it is functioning correctly.
    if (aes_ctr_prng_validate((aes_ctr_state_t*)*state) != 0)
    {
        nwipe_log(NWIPE_LOG_FATAL, "AES CTR PRNG validation failed.");
        aes_ctr_prng_general_cleanup((aes_ctr_state_t*)*state);
        free(*state);
        *state = NULL;
        return -1;
    }

    /* rest of the code here */
}

Actual validation.

/* Validates the pseudorandom data generation by generating test data and performing statistical tests.
   This function generates test data and performs:
   - Bit Frequency Test
   - Byte Frequency Test
   - Entropy Calculation
   - Checks for any obvious patterns
   - Thresholds are set to detect non-random behavior.
   - state: Pointer to the initialized AES CTR PRNG state.
   Returns 0 on success, -1 on failure. */
int aes_ctr_prng_validate(aes_ctr_state_t* state)
{
    assert(state != NULL);

    const size_t test_data_size = 4096;  // 4KB of data
    unsigned char* test_buffer = malloc(test_data_size);
    if (!test_buffer)
    {
        nwipe_log(NWIPE_LOG_ERROR, "Validation failed: Unable to allocate memory for test buffer.");
        return -1;
    }
    memset(test_buffer, 0, test_data_size);  // Zero out the buffer
    int outlen = 0;  // Length of data produced by encryption
    size_t total_generated = 0;

    // Generate 4KB of pseudorandom data
    while (total_generated < test_data_size)
    {
        size_t chunk_size = (test_data_size - total_generated);
        if (chunk_size > 512)
            chunk_size = 512;  // Generate data in chunks to avoid large allocations

        if (EVP_EncryptUpdate(state->ctx, test_buffer + total_generated, &outlen,
                              test_buffer + total_generated, chunk_size) != 1)
        {
            nwipe_log(NWIPE_LOG_ERROR,
                      "Failed to generate pseudorandom numbers during validation, error code: %s.",
                      ERR_error_string(ERR_get_error(), NULL));  // Log generation failure
            free(test_buffer);
            return -1;  // Handle error
        }
        total_generated += outlen;
    }

    // Proceed with statistical tests (unchanged from previous implementation)

    // ... [Rest of the validation code] ...

    free(test_buffer);
    return 0;  // Validation successful
}

Key Features:

  • AES-256-CTR Mode for strong security and unpredictability.
  • Integration with OpenSSL for robust cryptographic operations.
  • Automatic detection and utilization of AES-NI instructions for performance enhancement.
  • Dynamic linking of OpenSSL to benefit from security updates without recompilation.
  • Comprehensive validation step generating 4 KB of data and performing statistical tests:
    • Bit frequency test (±2% deviation allowed).
    • Shannon entropy calculation (minimum 7.5 bits per byte).
    • Pattern detection to identify anomalies.

Implementation Details:

  • Uses SHA-256 to derive a 256-bit key from the seed.
  • Initializes AES-256-CTR context via OpenSSL's EVP interfaces.
  • Generates pseudorandom data by encrypting sequential counter values.
  • Includes thorough error handling and resource management.

Statistical Test Results:

  • All tests in the NIST Statistical Test Suite passed.
  • TestU01 SmallCrush battery tests passed.
    • BigCrush battery tests passed.

This enhancement improves Nwipe's ability to meet stringent security requirements and provides a robust foundation for secure data wiping operations.
For further reference check #559

…th OpenSSL and AES-NI support

This commit introduces significant improvements to `nwipe`'s random number generation by implementing AES-CTR mode using OpenSSL with AES-NI hardware acceleration. The changes include:

- **AES-CTR PRNG Implementation**: A new pseudo-random number generator (PRNG) based on AES-128 in counter (CTR) mode has been added. This PRNG uses OpenSSL’s EVP API for cryptographic operations, ensuring strong random number generation.

- **AES-NI Hardware Support**: The PRNG now detects if the system supports AES-NI and uses hardware acceleration when available for improved performance, particularly on 64-bit systems. A new function `has_aes_ni()` checks for AES-NI support, ensuring that the fastest available option is used.

- **Default PRNG Selection**: For 64-bit systems, AES-CTR is now the default PRNG due to its higher performance and security. On 32-bit systems, the XORoshiro-256 PRNG remains the default due to performance considerations.

- **Improved Error Handling**: Error handling has been extensively revised. The AES-CTR PRNG now provides detailed error messages, including the detection of potential OpenSSL failures, and uses structured logging (`NWIPE_LOG_DEBUG` and `NWIPE_LOG_NOTICE`) to report PRNG initialization and errors.

- **Memory Management**: Smart pointers have been introduced to manage memory automatically within the AES-CTR PRNG, preventing memory leaks. All memory used by the OpenSSL contexts (EVP_MD_CTX and EVP_CIPHER_CTX) is now properly cleaned up after use.

- **Performance Enhancements**: The PRNG’s performance has been optimized, especially for 64-bit systems. Additionally, the code now checks for edge cases when reading remaining bytes, and memset is used to handle uninitialized memory areas.

- **Formatting and Code Cleanup**: Deprecated SHA256 calls have been replaced with OpenSSL’s EVP API. Multiple formatting issues in various source files (`pass.c`, `options.c`, `gui.c`, etc.) have been fixed, improving code readability and compliance with the coding standards.

- **Regression Fixes**: Several small regressions from previous changes have been fixed, including uninitialized variables, implicit function declarations, and incorrect function calls that previously led to warnings and potential crashes (e.g., segfaults in `pass.c`).

- **Logging and Debugging**: Improved logging messages and detailed debug information have been added, particularly around the AES-CTR initialization and error handling paths.

This implementation significantly improves the security, performance, and reliability of random number generation in `nwipe` and ensures that the tool is better equipped for modern cryptographic standards.

Squashed commit of the following:

commit 03284d7
Author: Fabian Druschke <[email protected]>
Date:   Fri Aug 23 19:17:05 2024 +0200

    Fixed formatting

commit 31df3eb
Author: Fabian Druschke <[email protected]>
Date:   Fri Aug 23 19:15:01 2024 +0200

    Fixed type error on i686 - uint64_t for AES-CTR

commit e56a47c
Merge: 3465260 d1edd05
Author: Fabian Druschke <[email protected]>
Date:   Tue Aug 20 10:39:26 2024 +0100

    Merge branch 'master' into aes-ctr

commit 3465260
Merge: 1bf5ff1 5140f92
Author: Fabian Druschke <[email protected]>
Date:   Fri May 10 01:41:09 2024 +0200

    Merge branch 'master' into aes-ctr

commit 1bf5ff1
Author: Fabian Druschke <[email protected]>
Date:   Fri May 10 01:14:11 2024 +0200

    Added check for return value in prng.c and nwipe_log accordingly.

commit 9015a32
Author: Fabian Druschke <[email protected]>
Date:   Fri May 10 00:11:59 2024 +0200

    Removed error handling using goto, now returning instead. Changed from void to int functions, removed cleanup(); on error , instead relying on cleanup routine on exit.

commit 10ea2bb
Author: Fabian Druschke <[email protected]>
Date:   Mon Apr 15 13:47:05 2024 +0200

    Adapted the aes-ctr-prng accordingly, to report SANITY level errors, in case of failure.

commit 71e7f8f
Author: Fabian Druschke <[email protected]>
Date:   Mon Apr 15 13:46:35 2024 +0200

    Added case handling for NWIPE_LOG_LEVEL_SANITY, providing the github issue link, if a SANITY level error occurs.

commit 0f3e7f5
Author: Fabian Druschke <[email protected]>
Date:   Mon Apr 15 00:40:22 2024 +0200

    Added check in pass.c wether nwipe_aes_ctr_prng is being used or not, if not a segfault was the result. Now fixed.

commit ce09d8e
Author: Fabian Druschke <[email protected]>
Date:   Sat Apr 13 00:58:34 2024 +0200

    Missing NWIPE_LOG_NOTICE for AES-CTR init changed to DEBUG level.

commit 8b284f3
Author: Fabian Druschke <[email protected]>
Date:   Sat Apr 13 00:50:25 2024 +0200

    Changed notification for successful PRNG init for AES-CTR to NWIPE_LOG_DEBUG.

commit 8702cc3
Author: Fabian Druschke <[email protected]>
Date:   Sat Apr 13 00:13:51 2024 +0200

    Added missing cleanup routine in prior commit in aes_ctr_prng.c and header definitions.

commit 35cd055
Author: Fabian Druschke <[email protected]>
Date:   Sat Apr 13 00:12:01 2024 +0200

    Fixed formatting in pass.c

commit beff746
Author: Fabian Druschke <[email protected]>
Date:   Sat Apr 13 00:11:28 2024 +0200

    Added cleanup routine aes_ctr_prng_general_cleanup() after nwipe_random_pass and nwipe_random_verify in order to cleanup PRNG state.

commit 1a95202
Author: Fabian Druschke <[email protected]>
Date:   Fri Apr 12 23:25:04 2024 +0200

    Part of the comments were missing, fixed.

commit a65410a
Author: Fabian Druschke <[email protected]>
Date:   Fri Apr 12 23:20:41 2024 +0200

    Added extensive error handling, in order to check for OpenSSL library malfunction.

commit 6518963
Author: Fabian Druschke <[email protected]>
Date:   Thu Apr 11 00:20:54 2024 +0200

    Improved PRNG description for AES-CTR-256 in gui.c

commit fe493cf
Author: Fabian Druschke <[email protected]>
Date:   Wed Apr 10 23:51:12 2024 +0200

    Added function has_aes_ni() in order to check for AES-Ni support, and set the PRNG accordingly.

commit adcd442
Author: Fabian Druschke <[email protected]>
Date:   Wed Apr 10 15:40:24 2024 +0200

    Handle edge case for remaining bytes in nwipe_aes_ctr_prng_read using memset.

commit ce2db63
Author: Fabian Druschke <[email protected]>
Date:   Wed Apr 10 15:08:25 2024 +0200

    Fixed comments, indicating AES-CTR-128 instead of 256 bit

commit da53ee0
Author: Fabian Druschke <[email protected]>
Date:   Wed Apr 10 14:53:45 2024 +0200

    Reverted by mistake nwipe_random back to nwipe_dodshort, now compliant with master

commit e479239
Author: Fabian Druschke <[email protected]>
Date:   Wed Apr 10 13:23:36 2024 +0200

    Fixed formatting

commit e0d9584
Author: Fabian Druschke <[email protected]>
Date:   Wed Apr 10 13:22:25 2024 +0200

    Minor changes, added comments for further explanation.

commit 7410d21
Author: Fabian Druschke <[email protected]>
Date:   Tue Apr 9 12:49:02 2024 +0200

    Fixed uninitialized temp_buffer in aes_ctr_prng_genrand_uint256_to_buf

commit cd5f071
Merge: 3cb78ca 2809580
Author: PartialVolume <[email protected]>
Date:   Sun Apr 7 22:22:55 2024 +0100

    Merge branch 'master' into aes-ctr

commit 3cb78ca
Author: Fabian Druschke <[email protected]>
Date:   Sun Mar 31 14:03:38 2024 +0200

    Added error checking and nwipe_log to aes_ctr_prng.c

commit 20fea0d
Author: Fabian Druschke <[email protected]>
Date:   Sun Mar 31 13:54:54 2024 +0200

    Only C implementation, removed CPP here.

commit d5b39f6
Author: Fabian Druschke <[email protected]>
Date:   Sun Mar 31 00:43:58 2024 +0100

    Introduced smart pointers to manage memory for EVP_MD_CTX and EVP_CIPHER_CTX within the AES CTR PRNG C++ implementation. This ensures automatic resource release, preventing memory leaks and enhancing code safety.

commit f6aeae3
Author: Fabian Druschke <[email protected]>
Date:   Sun Mar 31 00:34:15 2024 +0100

    Created seamless integrated .cpp AES-CTR-PRNG in order to avoid memory issues.

commit eecddb2
Author: Knogle <[email protected]>
Date:   Sun Mar 24 22:48:06 2024 +0000

    aes_ctr_prng_init was missing in header, causing implicit declaration warnings.

commit 8fe4db4
Author: Knogle <[email protected]>
Date:   Sun Mar 24 18:30:52 2024 +0000

    Replaced traditional deprecated SHA256 declarations with EVP-API infrastructure

commit 55472fb
Author: Knogle <[email protected]>
Date:   Sun Mar 24 03:43:36 2024 +0000

    To consider, AES-128-CTR as default option for 64-Bit, and Xoroshiro-256 as default option for 32-Bit due to performance and quality reasons.

commit 1a964bc
Author: Fabian Druschke <[email protected]>
Date:   Sat Mar 23 19:12:24 2024 -0300

    Fixed missing XORoshiro-256 in options.c bottom section. Added AES-128-CTR OpenSSL descriptions.

commit cf9822a
Author: Fabian Druschke <[email protected]>
Date:   Sat Mar 23 18:48:38 2024 -0300

    Several changes, adding AES-128 using libssl in CTR mode as new PRNG, in experimental state. Fixed formatting, fixed AES PRNG header.
- Introduce NWIPE_ASSERT macro to handle assertions with logging
  - Logs assertion failures using nwipe_log before aborting
  - Supports different log levels for flexibility
  - Respects NDEBUG flag to disable assertions in release builds

- Replace all existing assert statements with NWIPE_ASSERT
  - Enhanced error reporting with detailed log messages
  - Improved debugging capabilities by providing context on failures

- Update comments to English for consistency and broader accessibility

- Ensure inclusion of nwipe_assert.h where necessary
  - Facilitates reusability and modularity of the custom assert mechanism
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant