Skip to content

Use numpy string dtype to get accurate memory usage #307

Open
@jeromekelleher

Description

@jeromekelleher

We're currently using object dtypes in the BufferedArray class when dealing with string input and this gives us inaccurate numbers when we look at the array nbytes. To allow us understand where memory is being used (and perhaps use it better) we should switch to the new numpy StringDtype which should help.

So, at least we're buffering the data in an efficient format, and ideally this is basically what Zarr will use ultimately as well.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions