batch files cannot be used cross-platform

[kushalkolar]

I think it's because the current batch path is stored in DataFrame.attrs

mesmerize-core/mesmerize_core/batch_utils.py

Lines 59 to 60 in f00949f

	def set_batch_path(self, path: Union[str, Path]):
	self._data.attrs["batch_path"] = Path(path)

should just store this as a string and have a @batch_path property on the extension class which returns a Path instead of str to use within the extension functions.

[ethanbb]

Hey, I just ran into this issue as well.

I was thinking, your solution would avoid the error, but it's still not very useful to have the string representation of a path that's not usable on your current system. The batch path gets overwritten when loading a batch anyway right after un-pickling:

df = pd.read_pickle(Path(path))
df.paths.set_batch_path(path)

Thus how about this attribute just gets cleared before saving? Each df.to_pickle call would have to be replaced with a custom save function. If that sounds OK, I can take a stab at it.

[ethanbb]

By the way, how do you quote specific lines with the line numbers and link to file? I've been wishing there was a feature like that.

[kushalkolar]

not very useful to have the string representation of a path that's not usable on your current system. The batch path gets overwritten when loading a batch anyway right after un-pickling:

Ah yes you're right 😆 nice to have fresh eyes take a look at this

By the way, how do you quote specific lines with the line numbers and link to file? I've been wishing there was a feature like that.

https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-a-permanent-link-to-a-code-snippet

[ethanbb]

I just realized that the outputs are also saved as Paths, e.g.:

https://github.com/nel-lab/mesmerize-core/blob/38f6ebebf6dc38bc5f9312751f2848952711807e/mesmerize_core/algorithms/cnmf.py#L118C9-L134C10

Since I believe these are all relative paths, which should be cross-platform if they use forward slashes, I think the solution is to convert the relative paths to PurePosixPaths and then strings before saving.

Edit: or just save as PurePosixPath, that should also be fine.

[kushalkolar]

That's a good idea! Posix path and then string.