Can't update existing attributes in edit mode

[garrettmflynn]

When opening an existing file in edit ('a') mode, any attempt to update an existing attribute (or other aspects of the file, to my knowledge) results in internal warnings that keep the attribute unmodified. Is this the intended / known behavior of the library?

Here's a minimal example that will trigger these warnings (code, demo):

import h5wasm from "https://cdn.jsdelivr.net/npm/[email protected]/dist/esm/hdf5_hl.js";
const { FS } = await h5wasm.ready;

let new_file = new h5wasm.File("myfile.h5", "w");

const attrToUpdate = "new_attr"

new_file.create_attribute(attrToUpdate, "something wicked this way comes");
new_file.close()

let editable = new h5wasm.File("myfile.h5", "a");
editable.create_attribute(attrToUpdate, "something new this way comes");
editable.close()

let readable = new h5wasm.File("myfile.h5", "r");
console.log(`Got updated ${attrToUpdate} on hdf5 file:`, readable.attrs[attrToUpdate].value)

This may be related to #14, though this PR specifically flagged their need to update datasets—not other parts of the file.

Currently, webnwb loads all file contents into a JS object that tracks changes and is recompiled into a separate HDF5 file when the user decides to save them—though it would be significantly more elegant to place updates into the existing h5wasm File instance (loaded in 'a' mode) when that decision occurs and close the file instance when all edits are complete.

[bmaranville]

You're right - this is not a feature that has been implemented at all in h5wasm. When an h5wasm.File object is opened in "append" mode, that means you can add new datasets or attributes or groups, but no attempt has been made to allow you to edit existing objects.

This would take some research and work - it seems like if you're not changing the datatype or shape, you can just overwrite an attribute's data, but in the example you've given above the dtype is different between the old attribute value and the new. Looking at the source code for h5py for setting an attribute when it already exists, it seems they do a little dance where they create a temporary attribute from the new data, then delete the existing attribute, then rename the temporary to the old name. It's still easier than adding dataset editing like in #14

Have you considered using pyodide and h5py in your code? It takes a while to start up (~5 seconds on a newish computer) but after that you have the full h5py API in javascript. For advanced manipulations of HDF5 objects it might be worth the startup cost to get all that functionality.

[bmaranville]

It looks like it would be pretty easy to add a method

function delete_attribute(attr_name: string): number

to h5wasm, then in your code you could overwrite attrs by delete_attribute followed by create_attribute?

[garrettmflynn]

Thank you for your detailed response! Being able to delete properties would be really wonderful. Are there additional complexities with deleting groups, datasets, etc. in this way?

The library I’m trying to replicate in JavaScript (PyNWB) is dependent on h5py and I’ve tried using that library in Pyodide. However, I’d had some issues with enumeration and toJs resolution. This may, however, be due to my lack of familiarity with Pyodide—and I'll admit the exploration was quite quick. For a full rundown of what I'd learned from it, see Issue 4 on WebNWB.

[bmaranville]

Yeah, I have to admit that making detailed interconnects between python code and javascript code in pyodide still seems pretty complicated to me. I have some projects that mostly stay on the python side, or mostly on the javascript side, and that's easier than trying to mesh the two.

I added a delete_attribute method to the main branch of h5wasm, so it should be in the next release - how important is it to your use case to also be able to delete groups and datasets?

[garrettmflynn]

Thank you so much! Other edit/delete operations are much less important. Would be a nice to have—but definitely not a need-to have at the moment.

…

On Mon, Apr 10, 2023 at 7:24 AM, Brian Benjamin Maranville < ***@***.***> wrote: Yeah, I have to admit that making detailed interconnects between python code and javascript code in pyodide still seems pretty complicated to me. I have some projects that mostly stay on the python side, or mostly on the javascript side, and that's easier than trying to mesh the two. I added a delete_attribute method to the main branch of h5wasm, so it should be in the next release - how important is it to your use case to also be able to delete groups and datasets? — Reply to this email directly, view it on GitHub <#48 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALDAY5NQHITO5MGBOPVQGTLXAQJ3RANCNFSM6AAAAAAWW7KG5Y> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bmaranville]

Dataset.delete_attribute and Group.delete_attribute are implemented in the newest v0.4.11, published to npm and github just now. (also able to create resizable datasets and overwrite sections of data, and resize datasets)

[garrettmflynn]

Beautiful. Thanks again for your help.

…

On Wed, Apr 19, 2023 at 7:48 AM, Brian Benjamin Maranville < ***@***.***> wrote: Dataset.delete_attribute and Group.delete_attribute are implemented in the newest v0.4.11, published to npm and github just now. (also able to create resizable datasets and overwrite sections of data, and resize datasets) — Reply to this email directly, view it on GitHub <#48 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALDAY5ISD4QV3A5ZMFGK2Y3XB73L7ANCNFSM6AAAAAAWW7KG5Y> . You are receiving this because you authored the thread.Message ID: ***@***.***>