Backend Overview
This page should give an overview over the data structure and backend of the app. This is not a full description of all details and it never will be. It is more of a collection of important pieces of information and graphics and can be used to point to when talking about the system or as a reference when working on the app. If you are only looking for how to programmatically interact with the backend, refer to sections Database level and Practical usage (code).
The most relevant thread on why we ended up on this design can be found in Issue #239.
On a technical level all of the following elements are defined. Practically, on a business level, all classes, tables and enums serve the purpose of properly storing Tricks and Combos.
It is important to understand that the primary key of both tricks and combos is always a composition of their id and their "status". This status can be "official", "userDefined" or "archived". This means, that official and custom (and archived) tricks do not share the same id space. Two tricks can have the same id and only differ in being and official and a custom trick.
The meaning of Official and UserDefined should be clear: Official tricks are provided with the app, are defined by the developers and can not be altered or deleted by the user (except for the metadata). UserDefined tricks are created by the user and can be altered arbitrarily or be deleted by the user.
Archived is less obvious and exists for one special case: If an official trick or combo gets deleted and has previously existed in the users database it is not simply deleted. After all, the user might still want to have the trick / combo in their list even though it is not an official one anymore. Therefore, when an official trick is deleted by the developers, the status of the trick on the users devices is changed to "Archived". Then the user is prompted on whether the trick should be deleted or converted into a user defined trick.
When dealing with tricks and combos, all database transactions are hidden away behind Database Access Objects (DAOs) and the DbObject "interface". There are two DAOs in the project at the moment: the TricksDao and the CombosDao. Accordingly there are two classes implementing the DbObject "interface": Trick and Combo.
Fetching and the creation of tricks is done through the DAOs. When creating or fetching a trick / combo, they return a Trick or Combo object. As these implement the DbObject-"interface", they have to provide a persist method. This is used for persisting the object to the database, i.e. saving / updating. They further have to provide a delete method for deleting the trick or combo. In short: Create and Read through the DAOs, Updates and Deletes through the objects themselves.
Neither the DAOs nor the DbObjects protect against the deletion of official tricks.
Here is an example of how data is handled on a code-level. We want to load the official trick with the id 42, update the isFavourite attribute of the trick and then persist the change to the database. The changed attribute will be demonstrated along the way.
First we use the tricksDao to fetch the trick we want from the database.
import { tricksDao } from '../lib/database';
const entry = await tricksDao.getById(42, "official");Since the getById-function simply returns undefined if the trick is not in the database, we can do a simple check if the trick existed:
if(!entry) {
throw new Error("Doesn't exist!")
}Updating the isFavourite attibute as storing it is as simple as executing the following two lines:
entry.isFavourite = true;
await entry.persist()Attributes can be modified on the class instance by using the setters and persisted using the persist() method.
In order to check whether the attributes have been changed relative to the values in the database, the changed attribute can be used. To demonstrate this, consider the outputs of the console logs that were added to the previous code snippet. The outputs are based on the assumption that isFavourite is false in the database.
console.log(entry.changed) // -> false
entry.isFavourite = true;
console.log(entry.changed) // -> true
await entry.persist()
console.log(entry.changed) // -> falseAn overview of the data flow from the Google sheet can be seen in the following graphic. A detailed description can be found below.
The source of truth for the official tricks and combos is the Google Sheet. It can be edited only by @bastislack, while every other person can only make comments to recommend changes.
Both tricks and combos are fetched from the Google Sheet via a GitHub Action, which converts them into YAML files. It then stores them at /src/data/tricks and /src/data/combos and automatically opens a pull request, asking for approval of the update. This PR can then be merged by anyone with "Collaborator" rights or higher.
You can find the Action at .github/workflows/syncsheet.yml. The Action basically consists of two scripts:
-
scripts/fetchFromSheet.ts- can be seen as the "mapper" which converts data in the Google Sheet into YAML files. It takes a Sheet ID and 3 Sheet GIDs (=Table ids), downloads the Tables as TSV (like CSV, just using Tabs instead of Commas), validates the input, and then builds YAML Files for the entries in the Google Sheet. -
scripts/createPrForSheetsIfNeeded.ts- This is expected to run afterfetchFromSheet.ts. If any changes are detected, this script will create or update a Pull Request with the changed YAML files. A maintainer can then review the changes and approve them. If any errors are spotted they have to be changed in the Google Sheet.
Once the tricks are in the project as YAML files, they are then managed by a custom virtual module. This module exposes the content of the YAML files to the rest of the App as JS objects. It exposes all tricks, all combos and a hash based on all tricks and combos.
On startup the App checks if the hash stored in the user's local database is different from the one exposed by the virtual module. A difference in the hash indicates that the data for the official tricks and combos needs to be updated. This procedure is implemented here and more information on how updating works can be found in here in this article.
From there on out the App works as usual by making use of the local database.
Official tricks are updated in the Google Sheet and only there. When the developers decide that the changes should be added to the app, the GitHub action is triggered and the updates of the sheet are applied to the app in the form of updating the YAML files.
When the user updates the app, they receive the new official stickables (= tricks and combos) via the YAML files. This means that they still need to be updated in the user's database. This is fairly straight forward, except for when official stickables get removed from the app. In this case they get converted into "archived" stickables on the user side. The user gets prompted whether the stickables should be converted to custom ones or whether they should be deleted. The process can be seen in the following flowchart:
The flowchart displays the functional level but not necessarily the implementation on the technical level.