Parquet Data Loaders and complex data hierarchy

[GatsbyFitz]

Using a dataloader, I write the output of an API into a parquet file, I then read this in a md file.

buf = pa.BufferOutputStream()
table = pa.Table.from_pandas(df_out)
pq.write_table(table, buf, compression="snappy")

buf_bytes = buf.getvalue().to_pybytes()

sys.stdout.buffer.write(buf_bytes)

However, when I read it back in JS, the object seems to be massively more complex.

Its there a way to simplify the object hierarchy into the original dataset? As I am then looking to flatten the hierarchy so I can turn it into an Observable Plot tree?

Apologies if this is an obvious issue.

Many thanks,
Gatsby

[GatsbyFitz]

My solution, turn to JSON, then to JS Array:
`const data = arrow_table.toArray()

// Convert Arrow table to a JSON string

const jsonString = JSON.stringify(data); // Convert JSON string to a JavaScript array

const array = JSON.parse(jsonString);`