feat: add createdAt and updatedAt fields to public tables

[TristanWasTaken]

• [ x ] I agree to license my contribution under AGPL-3.0-only with my contribution automatically being licensed under LGPL-3.0 additionally after 6 months

---

Add a createdAt and updatedAt field.

Both fields would make incremental database updates easier and more efficient.
It would allow mirrors to selectively update small chunks at a time, instead of having to do a full table diff.

This is especially important for me personally, as I want to work on partial updates that don't rely on downloading the whole table.
(though it would still be difficult with deleted rows, since they would still need a full table diff. Soft deletes with a deletedAt or deleted(bool)+updatedAt field would probably be the best fix)

---

The changes:

• Add an updatedAt field to all public tables where I could find an UPDATE sql query in the code.
  • Tables that I couldn't find an update query for: thumbnailTimestamps, vipUsers, videoInfo, unlistedVideos, archivedSponsorTimes, ratings, thumbnails
• Add a createdAt field to all public tables that don't already have a timeSubmitted field

---

Notes:

• I currently only added the fields to tables that are available for download
  While it is nice that all necessary tables have the fields, I would like to add them to all tables. And yes, even the private db.
• For the type, etc. I just followed how timeSubmitted was implemented.
  • I was thinking of using pg's timestamp type and current_timestamp default value, but this would make it hard to switch to a different db if necessary.
  • It prevents anomalies from happening where timeSubmitted and updatedAt have a different source, making it possible for e.g. the createdAt timestamp from the pg server to be younger than timeSubmitted from the node server.

[TristanWasTaken]

Also, please tell me if I should update any of the tests
I didn't change any queries in the test folder for now, as some haven't been updated in quite some time.

[ajayyy]

thanks! this looks cool

tests need to pass. Probably possible to wrap things into a function to not have to copy and paste Date.now() everywhere in the tests. Also need to look into what broke the postgres GitHub action to make sure those tests pass with the changes, I think docker-compose command is now docker compose

[ajayyy]

What if we do this for createdAt?

https://stackoverflow.com/a/30686739

It's not 100% necessary for this function to work on SQLite, so maybe the schema could fallback to just defaulting to 0 for SQLite. There is a comment format you can use to exclude a line in the .sql schema file from being run on postgres or sqlite

[TristanWasTaken]

Also need to look into what broke the postgres GitHub action to make sure those tests pass with the changes, I think docker-compose command is now docker compose

yeah, it is

What if we do this for createdAt?

https://stackoverflow.com/a/30686739

It's not 100% necessary for this function to work on SQLite, so maybe the schema could fallback to just defaulting to 0 for SQLite. There is a comment format you can use to exclude a line in the .sql schema file from being run on postgres or sqlite

That'd also be possible. Besides my previously mentioned concerns of desynced time sources, if we're gonna go for storing the timestamp as a string, then I'd really like it to be in ISO 8601 (js: Date.toISOString(), pg: SELECT to_char(now()::timestamp AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.MS"Z');)

On that note, Falsehoods programmers believe about time is always a good read when working with timestamps 😄

[TristanWasTaken]

thanks! this looks cool

tests need to pass. Probably possible to wrap things into a function to not have to copy and paste Date.now() everywhere in the tests. Also need to look into what broke the postgres GitHub action to make sure those tests pass with the changes, I think docker-compose command is now docker compose

@ajayyy In the beginning I started work on another pr that would make updating the test queries a bit easier, but I got busy and ended up just updating the single queries 😔
Anyways, all affected queries should have been updated.

[WEI567]

Xx590731

[WEI567]

Xx590731

[WEI567]

Xx590731