.inlang folder for metadata

[ivanhofer]

Maybe we should think of adding some metadata files in a .inlang folder.

Use cases:

• tell what messages need to be reviewed (batch machine translate messages #328)
• general status what strings where machine translated
• cache lint results (Cache lint results #354)
• storing comments (support for description on messages #341)
• storing images (For translators: generate app screenshots for additional context #171)

The folder needs to be committed to the github repository. A .gitignore file inside that folder would make sure that the cached lint-result would not be committed.

If a message that is marked as machine-translated get's updated in the IDE (not editor), it should loose the machine-translated flag.
So a new sync command is needed for @inlang/cli that goes trough the last commits since the previous sync (information is stored in the .inlang folder ;)) and updates the metadata files.

I think the best solution would be to create a github plugin someone would need to install on the repository that executes this command on each push. If the project is hosted somewhere else the cli command can be used as a fallback, but inlang could provide a template for the most common plattforms.

[samuelstroschein]

This discussion is touching on a big question and decision: How are apps, that are being built on version control, store data that that should be stored in a database?

Using a directory like .inlang and committing files (no changes to git required)

Pros

• local data = every "front-end" (ide extension, inlang editor, etc.) will just work without auth/server coordination
• local data = organization are in control of the data -> lower adoption barrier -> lower engineering effort for inlang (less GDPR issues etc.)

Cons

• will pollute the commit history with no benefit (committing a comment is overkill)
• will bloat repositories. binary data like images in git is a bad idea. that said, people abuse git for it. github even uses https://git-lfs.com/ by default.
• performance is questionable
• HOW TO DEAL WITH MERGE CONFLICTS??

Modyfing git to become a proper backend

Related read: the next git thesis of inlang.

It seems that this entire discussion could be solved and a new generation of software could emerge could if:

• storing binary data in git is possible by default (built-in git LFS)
  No need for a storage bucket anymore. storing entire databases like sqlite becomes possible. massive breakthrough, tools like https://www.dolthub.com/ will become obsolete overnight. data science/machine learning gets the power of version control.

• lazy loading of files
  Otherwise, cloning repos with binary data will take ages to the size. We already have that problem with the inlang editor. Repos with a size of over 30MB take minutes to clone in the browser even though we only need <1MB of data from the repo.

• not every change needs to be committed
  Storing comments, entire databases like sqlite suddenly becomes feasible. Writing documents will become a more pleasent experience too.

• git provides real-time collaboration
  Reduced risk of merge conflicts + much better UX for specific use cases. Nobody will be able to build a better Figma that uses the proposed version control backend if that new Figma has no real-time collabroation in addtiojn to version control enabled async collaboration.

• built-in auth via auth as code (https://www.osohq.com/)
  Apps don't need an auth layer anymore. organization can configure auth as they please and need. Potentially revolutionizing here as well. If that auth layer is also able to hook into databases like sqlite ... absolute massive potential impact. We are already running into this issue authorization and file-based security #153

Yep, large undertaking but it's part of what inlang is. Localization infrastructure and a test-case for a backend based on files in someting like a git repository. Building inlang and having discussions like this one will entail what such a backend needs.

[samuelstroschein]

We might be able to hack git to avoid the need to commit every little metadata change by storing metadata in .git/inlang.

Need to test whether:

• git ignores .git
• additional data in .git is still sent to hosts like GitHub (the data is persistent).

[ivanhofer]

I think I have found a solution to this problem: https://git-scm.com/docs/git-notes

You can attach messages and files to each commit via a built in feature 🎉

[samuelstroschein]

outdated discussion. here is the new one #1021 (reply in thread)