Skip to content

How do I write bytes to a binary column? #1642

Answered by mattfysh
mattfysh asked this question in Q&A
Discussion options

You must be logged in to vote

This bug occurs only when using dtypes, either directly by passing in the parameter, or when retrieved from Glue

on

elif desired_type == "bytes":
df[col] = df[col].astype("string").str.encode(encoding="utf-8").replace(to_replace={pd.NA: None})

A bytes type in pandas is object and when the desired type is bytes - for some reason - the column is being converted to a string and re-encoded into utf-8 ... causing a bytes array of abc (or 61 62 63) to be stringified and then encoded as b'abc' (or 62 27 61 62 63 27)

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by mattfysh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
1 participant