Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save all user data in open file format (CSV) #35

Open
johnwdubois opened this issue Sep 7, 2018 · 1 comment
Open

Save all user data in open file format (CSV) #35

johnwdubois opened this issue Sep 7, 2018 · 1 comment
Labels
enhancement New feature or request

Comments

@johnwdubois
Copy link
Owner

johnwdubois commented Sep 7, 2018

Is your feature request related to a problem?
When the user invests time in creating new information in Rezonator (by marking Rez links between words, creating Stacks, etc.), this new information needs to be saved in a way that is well integrated with the existing corpus information (e.g. the original corpus files that the user imported).

Ideally, all the information (both new and old) should be made available to the user in a format that they can easily access, interpret, and edit outside of Rezonator. In addition, the newly created information should be integrated in a clear way with the old (corpus) information, to provide the greatest utility to the user.

Describe the solution you'd like
When exporting data from Rezonator, save it in a common open file format, such as:

  • spreadsheet format (CSV)
  • JSON

Make sure to include the following information:

  1. data indexed by Rezonator index values (to ensure data integrity, and clarity)
  2. data indexed by the orginal corpus index values (discoID, uID, unitStart, unitEnd, etc.), as much as possible (to allow data integration)
  3. new information created by the user in the Rezonator coding session (link source, link goal, timestamps, chains, etc.; see wiki re Link and Chain.)
  4. old information from the original imported files (word text, word transcription, kind, class, quality/manner, etc.)
  5. The number of columns to include will be large (e.g. 20 to 40 columns, or more). All information described in the wiki entry on Data Structures for Discourse Transcriptions, especially for Word, should be included in the file export.
  6. For CSV files, separate files will be needed for Word, Unit, Link, Chain, and so on. With JSON, a more unified approach may be possible.

Additional context
It would also be nice to allow users to re-import data back into Rezonator from their CSV (and/or JSON) file. However, this may be more difficult, and more risky, because it's not possible to fully control how the user modifies the external file. This may require some syntax checking on import.
Alternatively, in the early stages of development, it may be best to simply block (or limit) implementation of an Import function. Or only allow it for a completely unmodified file (e.g. with a known checkSum?).

@johnwdubois johnwdubois added the enhancement New feature or request label Sep 7, 2018
@johnwdubois johnwdubois changed the title Export user-generated data in open file format (CSV, JSON) Export user-generated data in open file format (CSV) Sep 7, 2018
@johnwdubois
Copy link
Owner Author

johnwdubois commented Jan 27, 2019

The exported CSV files have multiple problems; for the bug ticket, see #149 .

  1. In some cases, important information is missing (e.g. timestamps in the word and unit tables, etc.). Every column that appears in the gridView should be present in the exported CSV file.
  2. Some grids are missing entirely (e.g. Line, Search, Hit, Pick, vClique...). If it exists in the GridView grid, it should have its own CSV file.
  3. Sometimes, when a high decimal value appears (as in the GameMaker numerical codes for color values), these may appear in scientific notation (with "E+" etc). Apparently this is an illusion created by LibreOffice; the underlying data is still there, uncorrupted. So for now, it doesn't matter. (At some point in the future we may want to figure out how to get the external CSV viewer software to consistently display this as an unmodified number.)

@johnwdubois johnwdubois added the bug Something isn't working label Jan 27, 2019
@johnwdubois johnwdubois removed the bug Something isn't working label Feb 16, 2019
@johnwdubois johnwdubois changed the title Export user-generated data in open file format (CSV) Save all user data in open file format (CSV) Aug 3, 2019
@johnwdubois johnwdubois added this to Core Aug 22, 2023
@johnwdubois johnwdubois moved this to To do in Core Aug 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: To do
Development

No branches or pull requests

3 participants