Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Mac OS LF char for csv #517

Closed
matthewhegarty opened this issue Feb 21, 2022 · 5 comments
Closed

Support Mac OS LF char for csv #517

matthewhegarty opened this issue Feb 21, 2022 · 5 comments

Comments

@matthewhegarty
Copy link
Contributor

matthewhegarty commented Feb 21, 2022

Hi
I'm testing some changes on django-import-export and I notice an issue with LF endings for csv data, which was the format on older mac os distributions.

>>> import tablib
>>> sb = 'id,name,author_email\r1,Some book,[email protected]\r'
>>> tablib.import_set(sb, format='csv')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/matthew/.virtualenvs/django-import-export/lib/python3.9/site-packages/tablib/core.py", line 908, in import_set
    return Dataset().load(normalize_input(stream), format, **kwargs)
  File "/home/matthew/.virtualenvs/django-import-export/lib/python3.9/site-packages/tablib/core.py", line 414, in load
    fmt.import_set(self, stream, **kwargs)
  File "/home/matthew/.virtualenvs/django-import-export/lib/python3.9/site-packages/tablib/formats/_csv.py", line 44, in import_set
    for i, row in enumerate(rows):
_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

This can be fixed by adding the 'newline' kwarg to normalize_input():

def normalize_input(stream):
    """
    Accept either a str/bytes stream or a file-like object and always return a
    file-like object.
    """
    if isinstance(stream, str):
        return StringIO(stream, newline='')
    elif isinstance(stream, bytes):
        return BytesIO(stream)
    return stream
  • tablib: 3.2.0
  • python 3.9.10
@matthewhegarty
Copy link
Contributor Author

@claudep Is it possible we could get a new release with this fix in? We would need it for our forthcoming v3 release of django-import-export. I also notice that there was a build failure. Could you re-run the job to test whether it is something we need to fix?

@hugovk
Copy link
Member

hugovk commented Apr 1, 2022

Yes, it's about time for a release. We could wait a day or two to see if we can get #516 in?

What's your schedule for django-import-export?

I don't want to delay if you're ready to go now, we can always release this now and make a new release when that's ready, it's pretty easy to release with the automation.


The build is too old to restart, but here it is passing on my fork:

https://github.com/hugovk/tablib/actions/runs/2076613414

And here's a PR to add a button to allow us to manually trigger builds in the future: #519.

@matthewhegarty
Copy link
Contributor Author

What's your schedule for django-import-export?

No rush, this is for our major release which has been pending for a while, so we can certainly wait until you are ready.

@hugovk
Copy link
Member

hugovk commented Apr 9, 2022

#516 is still in progress, so I'll make a release today.

@hugovk
Copy link
Member

hugovk commented Apr 9, 2022

Released in 3.2.1! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants