Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rounding when two-bytes data is converted to one-byte data? #125

Open
cjunekim opened this issue Dec 2, 2022 · 1 comment
Open

Rounding when two-bytes data is converted to one-byte data? #125

cjunekim opened this issue Dec 2, 2022 · 1 comment
Assignees
Labels
core wishlist New core feature requests documentation Issues within the documentation enhancement Improvements on existing features help wanted Issues where contributions are appreciated a great deal

Comments

@cjunekim
Copy link

cjunekim commented Dec 2, 2022

It seems to truncate the value when converting two-bytes to one-byte data.

For example, pitchbend to cc. Suppose the pitchbend data incoming is 0x60 0x68 (LSB and MSB, in order). Its normalized value is(tested on Lua console):

> ((0x68 << 7) | 0x60&0x7f)/16383
0.81840932674113

When it is handed to CC, it is converted as,

> ((0x68 << 7) | 0x60&0x7f)/16383 * 127.0
103.93798449612

This value is handed to an unsigned int and the fractional part is discarded becoming 103, which, in turn, in hex is 0x67. It isn't 0x68, even though it was closer to 0x68, mathematically. It's because there is no rounding algorithm when a conversion from higher resolution to lower resolution occurs. I think this can cause some confusion and issues.

What do you think?

@cbdevnet
Copy link
Owner

Hm yes, not optimal in this case but hard to fix. Will think about it some more..

@cbdevnet cbdevnet added enhancement Improvements on existing features help wanted Issues where contributions are appreciated a great deal documentation Issues within the documentation core wishlist New core feature requests labels Jun 24, 2024
@cbdevnet cbdevnet self-assigned this Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core wishlist New core feature requests documentation Issues within the documentation enhancement Improvements on existing features help wanted Issues where contributions are appreciated a great deal
Projects
None yet
Development

No branches or pull requests

2 participants