Skip to content
This repository has been archived by the owner on May 18, 2024. It is now read-only.

Round-to-even bug in modp_dtoa2 #52

Open
jeroen opened this issue Dec 27, 2019 · 0 comments
Open

Round-to-even bug in modp_dtoa2 #52

jeroen opened this issue Dec 27, 2019 · 0 comments

Comments

@jeroen
Copy link
Contributor

jeroen commented Dec 27, 2019

We found that modp_dtoa2 has odd cases where it does not properly round-to-even. For example x.665 should be rounded down to x.66 but sometimes it gets rounded to x.67

modp_dtoa2(9.8665, digits = 3)
# 9.867
modp_dtoa2(1.8665, digits = 3)
# 1.866

Or

modp_dtoa2(111.665, digits = 2)
# 111.67
modp_dtoa2(222.665, digits = 2)
# 222.66 
modp_dtoa2(333.665, digits = 2)
# 333.67
modp_dtoa2(444.665, digits = 2)
# 444.67
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant