float('inf') not supported on OEP

[henhuy]

Hi @MGlauer,

A colleague tries to upload float('inf') into OEP.
Normally, this is supported by postgresql type double precision and it works locally.

The following code works with my local postgresql, but fails with OEDialect (engine='postgresql+oedialect'):

import pandas
import sqlalchemy
import oedialect
from sqlalchemy import Column, BIGINT
from sqlalchemy.dialects.postgresql import DOUBLE_PRECISION
from sqlalchemy.ext.declarative import declarative_base

# engine connects to OEP correctly

Base = declarative_base()


class InfTest(Base):
    __tablename__ = 'inf_test'
    __table_args__ = {'schema': 'sandbox'}

    index = Column(BIGINT, primary_key=True)
    value = Column(DOUBLE_PRECISION)


Base.metadata.bind = engine

normal_df = pandas.DataFrame({'value': [4.3, 2.5]}).to_sql(
    'inf_test', engine, schema='sandbox', if_exists='append')  # This works 

inf_df = pandas.DataFrame({'value': [float('inf'), 2.5]}).to_sql(
    'inf_test', engine, schema='sandbox', if_exists='append')  # this fails

Error message from json_details is:

<class 'dict'>: {'detail': "JSON parse error - Out of range float values are not JSON compliant: 'Infinity'"}

As I know, OEP runs on postgresql server - could you allow for uploading 'Infinity' floats as well?

[jh-RLI]

@MGlauer Is this easy to fix? Like just add the 'Infinity' to a whitelist or something like that?

[MGlauer]

It is a bit more difficult. The communication format is JSON, which just uses the string representation "Infinity". For the API there is no way to distinguish this from the actual string "Infinity". Therefore, we have to do something more elaborate and extend the API to make those entities distinguishable (as is done already with different date formats).

[MGlauer]

This seems even more complicated: Pythons json-encode uses a extension of the json standard and this extension is more or less hard-coded in the JSONEncoder class. It seems like the only viable solution is for us to write our own version of JSONEncoder._iterencode.

[jh-RLI]

If we want to implement this, could you estimate the time needed for the implementation? @MGlauer

[MGlauer]

Already done. See ec6c8cb

[jh-RLI]

oh have not refreshed the page for a while :D good job

[christian-rli]

Already done. See ec6c8cb

Yay! Also: why is there no PR for this, or am I missing something?

[jh-RLI]

It seems like we need to update the master branch and change the default branch to develop.
@christian-rli I will create the missing PR. in #33 previous commits are added automatically I need
to figure this out first.