Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDLC performance degradation due to persistence format strategy #411

Open
4 tasks
akphi opened this issue Apr 11, 2022 · 0 comments
Open
4 tasks

SDLC performance degradation due to persistence format strategy #411

akphi opened this issue Apr 11, 2022 · 0 comments

Comments

@akphi
Copy link
Contributor

akphi commented Apr 11, 2022

SDLC (as of project structure version 8) starts to store entities in grammar text. This has many upsides but somewhat punish performance heavily.

  • In terms of write, it does multi-parsing and composing: for example, it, first, tries to ask if an entity is parsable, it composes then parses the entity and do a roundtrip comparison to make sure parsing is properly done. If yes, it proceeds with parsing, otherwise, it stores the entity in JSON, and this means when we read the entity, we have to look up where the entity could have been stored based on its format (please fact-check this).
  • In terms of read, it parses the entity from text to JSON, which doubles up the size due to source information being returned as well. Also because of this, entities being published to metadata server (a.k.a depot) also contains source information which really blows up the size of models to 100% if not 200%!

Therefore, we could consider:

  • Introducing a mechanism for SDLC server to prune source information while parsing grammar text
  • Introduce a mechanism for SDLC to store entities in JSON only, maybe a config at project structure level.
@akphi akphi changed the title SDLC performance degradation due to text storage strategy SDLC performance degradation due to persistence format strategy Apr 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant