Skip to content

Use getrandom on Linux for Guid generation for a 12% speed boost. #111287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from

Conversation

vcsjones
Copy link
Member

Since Linux 3.17, the syscall getrandom can be used to generate bytes from the same entropy source as /dev/urandom. This makes it suitable for cryptographic random number generation, something our Guid generator needs.

If getrandom is not available because the LibC provided is too old - or if the syscall itself is not available, it will fall back to the previous method of reading from /dev/urandom.

EPERM is treated like ENOSYS - some older Linux kernels returned this when the syscall is not available. The syscall can also be blocked by seccomp. That would be weird, and Docker permits it by default, but if anyone has explicitly gone out of their way to block it, it falls back to using the file descriptor mechanism.

The use of the syscall implementation instead of the file device offers some performance improvements. Anywhere from 10% to 15% in benchmarks.

This does require glibc 2.25. For non-portable builds, that's largely fine. For Microsoft builds, those are (currently) being done on Ubuntu Xenial, which has 2.23. However, it is expected that with .NET 10, the glibc floor is going to change to 2.27 with #109939. When that happens, then the Microsoft portable builds will pick this up as well.

This PR does not change the user-facing RandomNumberGenerator that will continue to use RAND_bytes from OpenSSL.

Method Toolchain Mean Error StdDev Ratio
NewGuid branch 265.1 ns 0.24 ns 0.22 ns 0.87
NewGuid main 304.6 ns 0.16 ns 0.15 ns 1.00
CreateVersion7 branch 293.2 ns 0.20 ns 0.18 ns 0.88
CreateVersion7 main 335.0 ns 0.27 ns 0.23 ns 1.00

Since Linux 3.17, the syscall `getrandom` can be used to generate bytes from the same entropy source as `/dev/urandom`. This makes it suitable for cryptographic random number generation, something our Guid generator needs.

If `getrandom` is not available because the LibC provided is too old - or if the syscall itself is not available, it will fall back to the previous method of reading from `/dev/urandom`.

`EPERM` is treated like `ENOSYS` - some older Linux kernels returned this when the syscall is not available. The syscall can also be blocked by seccomp. That would be weird, and Docker permits it by default, but if anyone has explicitly gone out of their way to block it, it falls back to using the file descriptor mechanism.

The use of the syscall implementation instead of the file device offers some performance improvements. Anywhere from 10% to 15% in benchmarks.

This does require glibc 2.25. For non-portable builds, that's largely fine. For Microsoft builds, those are (currently) being done on Ubuntu Xenial, which has 2.23. However, it is expected that with .NET 10, the glibc floor is going to change to 2.27 with dotnet#109939. If that happens, then the Microsoft portable builds will pick this up as well.

This PR does _not_ change the user-facing `RandomNumberGenerator` that will continue to use `RAND_bytes` from OpenSSL.

| Method         | Toolchain | Mean     | Error   | StdDev  | Ratio |
|--------------- |---------- |---------:|--------:|--------:|------:|
| NewGuid        | branch    | 265.1 ns | 0.24 ns | 0.22 ns |  0.87 |
| NewGuid        | main      | 304.6 ns | 0.16 ns | 0.15 ns |  1.00 |
|                |           |          |         |         |       |
| CreateVersion7 | branch    | 293.2 ns | 0.20 ns | 0.18 ns |  0.88 |
| CreateVersion7 | main      | 335.0 ns | 0.27 ns | 0.23 ns |  1.00 |
Copy link
Member

@jkoritzinsky jkoritzinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we extract the new code-path into a static function that returns true if getrandom was used and false if the device random should be used? I'd like to get rid of the goto if we can.

Comment on lines 111 to 113
static bool sMissingGetRandomSysCall = false;

if (!sMissingGetRandomSysCall)
Copy link
Member

@am11 am11 Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
static bool sMissingGetRandomSysCall = false;
if (!sMissingGetRandomSysCall)
static short sGetRandomSupport = -1;
if (sGetRandomSupport == -1)
{
sGetRandomSupport = syscall(SYS_getrandom, nullptr, 0, GRND_NONBLOCK) == -1 && errno == ENOSYS ? 0 : 1;
}
if (sGetRandomSupport == 1)

This run-time introspection will only run once; the first time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this better than using the getrandom provided by the C runtime?

Copy link
Member

@jkotas jkotas Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direct syscall has a small advantage in that it does not introduce hard dependency on the new glibc version, and so the binaries work in practice on older baselines too.

For example, even though our current baseline is Ubuntu 16, the binaries work in practice on older baselines too since we do not actually have a hard dependency on anything from Ubuntu 16 (it has been verified for native AOT compiled binaries, I am not sure about full coreclr).

@am11
Copy link
Member

am11 commented Jan 10, 2025

This looks good. I missed that you already handled ENOSYS. :)

It will also improve #31271 since it also ends up calling minipal implementation.

@vcsjones, I posted a benchmark with managed impl. here, if case you want to try out #31271 (comment).

@jkotas
Copy link
Member

jkotas commented Jan 27, 2025

This does require glibc 2.25. For non-portable builds, that's largely fine. For Microsoft builds, those are (currently) being done on Ubuntu Xenial, which has 2.23.

This means that we have close to non-existing test coverage for this change. We do nearly all our testing in this repo on portable builds.

We should either wait with merging this until we have updated the baseline, or define the syscall locally for portable builds like we have done in similar situations in the past see e.g. #111846 .

@vcsjones
Copy link
Member Author

I will take a look at doing a local definition.

@vcsjones vcsjones marked this pull request as draft February 1, 2025 14:40
Copy link
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actions github-actions bot locked and limited conversation to collaborators Apr 3, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants