Initial draft of harp file standard

[bruno-f-cruz]

This PR includes a first draft of the standardized harp file format

[glopesdev]

@bruno-f-cruz Thanks, will review soon. Are we changing the recommended casing from Harp to harp? I don't necessarily mind, but whichever way we decide to go, I would like us to be consistent across all documents. I think all references to the original upper-case HARP have now been replaced with Harp.

Note that calling the protocol "Harp" does not mean we need to exclude having CLI-tools and URL use harp, so we can still have harp-tech and harp.toolkit. This is similar to "Python" and "Git" in which the CLI and URL is spelled python and git, respectively, but across all docs online we see it cased as "Python" and "Git".

Maybe we can create a separate proposal for this if we want to change?

[bruno-f-cruz]

I may be confused but I thought that convention after the discussion for the sfn poster was to lower-case everything. That being said I am also ok with either and we should refactor the docs for sure. Does anyone else have a preference? @artursilva0 @filcarv @jfrazao maybe?

[artursilva0]

I thought we had already settled on the Harp a while back. One downside is that the logo is also in lowercase, but I'm ok with either option.

[glopesdev]

One downside is that the logo is also in lowercase, but I'm ok with either option.

@artursilva0 this is not inconsistent with other projects. No need to look further than Git. Logo in lower-case, all references in docs and everywhere with capitalized first letter.

[glopesdev]

I may be confused but I thought that convention after the discussion for the sfn poster was to lower-case everything.

I do remember hearing something about a preference for lower-case, but I don't remember the decision and like @artursilva0 I thought the matter was decided long ago. I think the cost of changing everything now would be complex and create even more confusion since we already used HARP and Harp, and introducing harp in docs just creates even more combinations.

[bruno-f-cruz]

All relevant references to Harp have now been capitalized in the text. Other references were left as lower case where appropriate (e.,g. paths and URLs)

[glopesdev]

Looks good to me, just minor comments for clarity and to try and accommodate the various ways in which we have been using the de-multiplexed file format.

[glopesdev]

Suggested change

	📦<Device>.harp
	📦<Device>

Do we really need the pseudo-extension .harp on these folders? I agree in the future we might want to evolve this convention into a specific file format which stores this hierarchy into something like a zipped file which can have this extension, but I don't see a particular advantage in placing constraints in folder names.

Of course, each lab / institution is free to establish their own conventions, I just feel it might be overly prescriptive at this point.

[bruno-f-cruz]

Sure, I still think that having a "sane default" is a good thing :P. Makes people's life easier when sharing datasets. But in the context of logging a single harp device I agree is overly-opinionated.

[glopesdev]

Suggested change

- `<DeviceName>` should match the `device.yml` metadata file that fully defines the device and can be found in the repository of each device ([e.g.](https://raw.githubusercontent.com/harp-tech/device.behavior/main/device.yml)). This file can be seen as the "ground-truth" specification of the device. It is used to automatically generate documentation, interfaces and data ingestion tools. While this is not a strict requirement, it is highly recommended.

- `<DeviceName>` should match the name in the `device.yml` metadata file that fully defines the device and can be found in the repository of each device ([e.g.](https://raw.githubusercontent.com/harp-tech/device.behavior/main/device.yml)). This file can be seen as the "ground-truth" specification of the device. It is used to automatically generate documentation, interfaces and data ingestion tools. While this is not a strict requirement, it is highly recommended.

Do we need to require that <DeviceName> matches the device type, or is it enough to match the device folder? In our current standards we tend to have individual files inside container folders be named following the full hierarchy structure leading up to the file.

This is useful in cases where there may be other types of files stored in the experimental device folder which are not harp binary files themselves, but are data stored and synched together with this device (e.g. video frame data).

In this case if we change the convention for non-Harp files the naming convention will be inconsistent. Alternatively, if we do change the convention to name it after the Harp device type, the naming convention may be misleading, e.g. consider the following:

CameraTop
 ┣ 📜Behavior_0.bin
 ┣ 📜Behavior_1.bin
 ┗ 📜CameraTop_Video.avi
CameraSide
 ┣ 📜Behavior_0.bin
 ┣ 📜Behavior_1.bin
 ┗ 📜CameraSide_Video.avi

If we do want to include the device type in the name I guess we could add the container name as a prefix and keep the harp device type in the file:

CameraTop
 ┣ 📜CameraTop_Behavior_0.bin
 ┣ 📜CameraTop_Behavior_1.bin
 ┗ 📜CameraTop_Video.avi
CameraSide
 ┣ 📜CameraSide_Behavior_0.bin
 ┣ 📜CameraSide_Behavior_1.bin
 ┗ 📜CameraSide_Video.avi

There are definitely other possible alternatives such as nested folders, etc, but as above I feel somehow we should avoid being overly prescriptive. It is true that the current harp-python currently adheres to a specific structure, but perhaps we could make it flexible to adhere to a variety of options.