Skip to content

Exact float conv #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

adegert
Copy link
Contributor

@adegert adegert commented Nov 19, 2019

An alternative to the current algorithm + new function rational_limit_denominator

Andreas Degert added 3 commits November 16, 2019 10:04
Overflow leads to wrong results and can create non-terminating loops
in the postgresql backend.

Changes:

 - return ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE when input is too large
   to be represented as rational

 - end refinement when the resulting numerator or denominator would
   not fit into an int32.

 - added some test cases
 - use double negation because conditions with NaN's always evaluate to false

 - add test case with NaN

 - some cosmetic changes
This version of rational_in_float() converts the float into an
int64/int64 rational and calculates the int32/int32 rational with the
least error (where possible reproducing it exactly) using continued
fractions.

Additionally a funktion rational_limit_denominator() is definined
which finds the best approximation for a rational with the denominator
not exceeding a given value. One use case is to recover fractions that
are stored with limited precision:

rational_limit_denominator(0.3333::float, 1000) -> 1/3
@begriffs
Copy link
Owner

Can you tell me more about this algorithm? What does it improve, and does it have any disadvantages compared with the existing one?

/*
limit_denomintor() uses continued fractions to convert the
rational n/d into the rational n'/d' with d' < max_denominator
and n' <= INT32_MAX and smallest |d/n-d'/n'|.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a paper or a description of this algorithm you can link to in the comment?

@adegert
Copy link
Contributor Author

adegert commented Nov 26, 2019 via email

@begriffs
Copy link
Owner

begriffs commented Dec 6, 2019

Thanks for suggesting this alternate way to do it. I merged your other PR because the changes are smaller, but we can discuss which way to do it on the pg hackers list.

In the meantime I'll be releasing another version of the extension to get your fix out there.

@begriffs
Copy link
Owner

Could you rebase this PR please?

@robertlagrant
Copy link

@adegert :)

@begriffs
Copy link
Owner

@adegert are you still interested in continuing development on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants