Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: make buflen in integer range #51821

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

kylo5aby
Copy link
Contributor

@kylo5aby kylo5aby commented Feb 21, 2024

make buflen within the integer range when buffer.toString() and writing a string to buffer(for example buffer.write(str))

Fixes: #51817

@nodejs-github-bot nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Feb 21, 2024
src/node_buffer.cc Outdated Show resolved Hide resolved
@kylo5aby kylo5aby force-pushed the buffer-utf8value branch 3 times, most recently from 3833a0f to d51e680 Compare February 22, 2024 05:06
Comment on lines 727 to 728
if (max_length > static_cast<size_t>(v8::String::kMaxLength))
ThrowErrStringTooLong(env->isolate());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (max_length > static_cast<size_t>(v8::String::kMaxLength))
ThrowErrStringTooLong(env->isolate());
if (max_length > static_cast<size_t>(v8::String::kMaxLength)) {
ThrowErrStringTooLong(env->isolate());
return;
}

You're missing a return here; the Throw* functions don't actually return, you have to do it manually.

Comment on lines 1158 to 1173
// Invalid length of Buffer.utf8Write
{
const ubuf = Buffer.allocUnsafeSlow(2 ** 32);
assert.throws(() => {
ubuf.utf8Write('a', 2, kStringMaxLength + 2);
}, stringTooLongError);
}

// Invalid length of Buffer.write
{
const ubuf = Buffer.allocUnsafeSlow(2 ** 32);
assert.throws(() => {
ubuf.write('a', 2, kStringMaxLength + 2);
}, stringTooLongError);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test probably won't work in test/parallel, since 2 GiB is a lot to allocate (maybe it'll work on Linux)? It might work to put it in test/sequential instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @kvakil

Copy link
Member

@joyeecheung joyeecheung Feb 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For tests that require a lot of resource, we typically use test/pummel. You could also catch ERR_MEMORY_ALLOCATION_FAILED in the buffer creation and common.skip() in the test if the memory could not be allocated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks your suggestion @joyeecheung , I have update it, add a method in StringBytes to ensure that the capacity of buffer remains within the integer range when operating with string

@kvakil kvakil added semver-major PRs that contain breaking changes and should be released in the next major version. request-ci Add this label to start a Jenkins CI on a PR. labels Feb 22, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 22, 2024
@nodejs-github-bot
Copy link
Collaborator

@kylo5aby kylo5aby changed the title buffer: fix Buffer.utf8Write fails when length exceeds integer range buffer: make ·buflen` in integer range Feb 22, 2024
@kylo5aby kylo5aby changed the title buffer: make ·buflen` in integer range buffer: make buflen in integer range Feb 22, 2024
@kylo5aby kylo5aby force-pushed the buffer-utf8value branch 3 times, most recently from 250acee to 2e93994 Compare February 23, 2024 05:48
src/string_bytes.h Outdated Show resolved Hide resolved
src/string_bytes.cc Outdated Show resolved Hide resolved
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 1, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 1, 2024
@nodejs-github-bot
Copy link
Collaborator

@kylo5aby
Copy link
Contributor Author

this PR has been approved but hasn't been landed for a long time, could you please take a look? @legendecas

@kylo5aby kylo5aby force-pushed the buffer-utf8value branch 2 times, most recently from 5b60d83 to 69cee50 Compare March 27, 2024 02:29
@legendecas legendecas added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 27, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 27, 2024
@nodejs-github-bot
Copy link
Collaborator

@kylo5aby
Copy link
Contributor Author

I find some test checks didn't pass, I believe that's because in the test file, the buffer with size > INT32_MAX on x32 platform which will throw range error, I have updated and skip the test file on x32 platform, could you take a look? @legendecas

@legendecas
Copy link
Member

This needs a rebase.

@kylo5aby
Copy link
Contributor Author

This needs a rebase.

Resolved. Thanks!

@legendecas legendecas added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label May 27, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 27, 2024
@nodejs-github-bot
Copy link
Collaborator

@theanarkh theanarkh added the request-ci Add this label to start a Jenkins CI on a PR. label May 30, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 30, 2024
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@jakecastelli
Copy link
Member

This will need a rebase again 😞

Copy link

codecov bot commented Aug 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.34%. Comparing base (0b3ae01) to head (db80e6e).
Report is 654 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #51821      +/-   ##
==========================================
+ Coverage   87.32%   87.34%   +0.01%     
==========================================
  Files         649      650       +1     
  Lines      182570   182847     +277     
  Branches    35026    35071      +45     
==========================================
+ Hits       159432   159701     +269     
- Misses      16406    16411       +5     
- Partials     6732     6735       +3     
Files with missing lines Coverage Δ
src/string_bytes.cc 66.76% <100.00%> (+1.25%) ⬆️

... and 61 files with indirect coverage changes

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 4, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 4, 2024
@nodejs-github-bot
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. semver-major PRs that contain breaking changes and should be released in the next major version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Buffer.utf8Write() fails to write when buffer length exceeds 2 GB
9 participants