Add docker to all components. #1

sennav · 2025-01-21T00:20:51Z

Hey Katie,

First of all, thank you for the presentation, I saw it on Youtube a few days ago and got inspired to try out myself. It would be really fun to build an engine in Rust that could be like a lambda runner and that's what I plan to try out in the future. I just have to learn from your code and learn more Rust before (pretty much a Rust n00b here 🐣).

While running your code I made some modifications to make it easier to run it locally, I pretty much added everything to containers so you can just run docker compose -f docker-compose.yaml up and it should just work™. I haven't changed any documentation yet since maybe you'll have feedback on the code and I'd have to change the code plus documentation, and I'm lazy. I can make another commit with the docs later. There are some things I am unsure if you want to keep separate somehow since you run this code on some server, I'm mainly thinking about having the certs on the backend image, maybe there could be production image and a local dev one, not sure about that.

I also have been considering changing the game to the prisioner's game, Derek Muller from Veritassium made a really nice video about it and this seems like a perfect project to implement bots for it (it also goes well with the title of your presentation, maybe 😅). Ofc this could be a separate repo altogether.

Let me know if you have any questions or feedback, have a good one.

* Introduce a postgres docker container. * Introduce a docker container to the client (via nginx). * Introuce a docker container with minio to replace S3 in a local setup. * Creates a script to generate a dummy cert file (satisfies the image generation requirement but not used on local development due to missing env variables).

sennav · 2025-01-21T00:22:16Z

client/package.json

@@ -6,7 +6,7 @@
  "dependencies": {
    "@emotion/react": "^11.11.1",
    "@emotion/styled": "^11.11.0",
-    "@g-loot/react-tournament-brackets": "^1.0.30",


Seems to be deprecated.

sennav · 2025-01-21T00:25:27Z

client/package.json

+  "packageManager": "[email protected]",
+  "overrides": {
+    "react-tournament-brackets": {
+      "styled-components": "$styled-components"


I believe I had to bump styled-components for some reason or the npm version I'm using is detecting that it conflicts with the version styled-components required from react-tournament-brackets. This seems to work anyway.

Yeah, seems to be a version mismatch in the devdependencies not the dependencies that matter. I might look into resolving this but it's fine. :)

sennav · 2025-01-21T00:26:31Z

wasi-runner/generate_database_cert.sh

+# Clean up the temporary files
+rm -f "$KEY_FILE" "$CERT_FILE"
+
+echo "Certificate and key have been successfully combined into '$OUTPUT_FILE'."


Just generates a dummy cert to satisfy the need for a cert in the backend Dockerfile.

Appreciate you going to the effort here to maintain the existing Dockerfile, but the cert shouldn't be needed for local dev, it's only there for the prod version because I deploy to AWS RDS.

If we modify the Dockerfile to look like this (replace three COPY lines with two):

COPY --from=builder /app/target/release/wasi-runner ./ COPY --from=builder /app/*.wasm /app/database_cert*.pem ./

Then it'll copy the database_cert.pem if it's there and ignore it if it's not and you shouldn't need to generate a cert. (If you prefer to use this generate script then that's also fine, happy to accept it, I just think it's unnecessary).

Also... just a quick side note, you may have noticed that the docker build is hella slow. It wasn't written for local dev, so it's not broken down into nice layers that cache well. I imagine rebuilding it each time you make a change takes a looong time. I can look into making it faster if that'd be helpful to you.

Nice, I removed it as you suggested. Yeah I noticed it is pretty slow too it will be nice to speed that up.

sennav · 2025-01-21T00:27:30Z

wasi-runner/local_setup.sql

@@ -1,7 +1,4 @@
 DROP TABLE IF EXISTS bots;
-DROP USER IF EXISTS snippyuser;
-
-CREATE USER snippyuser WITH PASSWORD 'snippy123';


This might disrupt some of your flow but with the containerized version of postgres the user gets created via env variables.

I have a mild preference to keep snippyuser and the root postgres user separate from each other (so that the permissions for snippyuser can be kept to a minimum), but honestly, either way is fine. :)

Good point, adopted your suggestion.

sennav · 2025-01-21T00:28:05Z

wasi-runner/src/tournament.rs

+        _ => {
+            S3Client::new(&shared_config)
+        }
+    };


Uses minio for development purposes instead of S3.

katharosada · 2025-01-21T00:49:42Z

Oh, this looks great, thats a big improvement! Thank you so much for contributing this! I'll give it a go on my machine to check it out.

katharosada · 2025-01-21T00:51:10Z

client/package-lock.json

I've been using yarn so ideally there shouldn't be an npm-style package-lock.json, just the yarn.lock file. :)

I was confused by the client README which mentioned npm, I've changed all to yarn now.

katharosada · 2025-01-21T00:51:38Z

client/Dockerfile

+COPY ./ .
+
+RUN npm install
+RUN npm run build


Could we change this to use yarn install and yarn build?

katharosada · 2025-01-21T01:11:17Z

wasi-runner/generate_database_cert.sh

+# Clean up the temporary files
+rm -f "$KEY_FILE" "$CERT_FILE"
+
+echo "Certificate and key have been successfully combined into '$OUTPUT_FILE'."


Appreciate you going to the effort here to maintain the existing Dockerfile, but the cert shouldn't be needed for local dev, it's only there for the prod version because I deploy to AWS RDS.

If we modify the Dockerfile to look like this (replace three COPY lines with two):

COPY --from=builder /app/target/release/wasi-runner ./ COPY --from=builder /app/*.wasm /app/database_cert*.pem ./

Then it'll copy the database_cert.pem if it's there and ignore it if it's not and you shouldn't need to generate a cert. (If you prefer to use this generate script then that's also fine, happy to accept it, I just think it's unnecessary).

Also... just a quick side note, you may have noticed that the docker build is hella slow. It wasn't written for local dev, so it's not broken down into nice layers that cache well. I imagine rebuilding it each time you make a change takes a looong time. I can look into making it faster if that'd be helpful to you.

katharosada · 2025-01-21T01:13:00Z

wasi-runner/local_setup.sql

@@ -1,7 +1,4 @@
 DROP TABLE IF EXISTS bots;
-DROP USER IF EXISTS snippyuser;
-
-CREATE USER snippyuser WITH PASSWORD 'snippy123';


I have a mild preference to keep snippyuser and the root postgres user separate from each other (so that the permissions for snippyuser can be kept to a minimum), but honestly, either way is fine. :)

Adds react-svg-pan-zoom to dependencies as it was not being resolved as a dependency of react-tournament-brackets.

Also improves startup sequence in the docker compose setup.

katharosada · 2025-01-22T09:00:22Z

This is great, I've tried it out locally and it all works for me. :) Definitely need to make that docker build faster though, oof.

sennav commented Jan 21, 2025

View reviewed changes

katharosada reviewed Jan 21, 2025

View reviewed changes

sennav added 3 commits January 21, 2025 12:54

Remove npm in favour of yarn

6a72981

Adds react-svg-pan-zoom to dependencies as it was not being resolved as a dependency of react-tournament-brackets.

Remove dummy certificate generation

296d82b

Separates postgres root user from application user

2b93bb1

Also improves startup sequence in the docker compose setup.

katharosada merged commit 0ecfc11 into katharosada:main Jan 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docker to all components. #1

Add docker to all components. #1

sennav commented Jan 21, 2025 •

edited

Loading

sennav Jan 21, 2025

sennav Jan 21, 2025

katharosada Jan 22, 2025

sennav Jan 21, 2025

katharosada Jan 21, 2025

sennav Jan 21, 2025

sennav Jan 21, 2025

katharosada Jan 21, 2025

sennav Jan 21, 2025

sennav Jan 21, 2025 •

edited

Loading

katharosada commented Jan 21, 2025

katharosada Jan 21, 2025

sennav Jan 21, 2025

katharosada Jan 21, 2025

sennav Jan 21, 2025

katharosada Jan 21, 2025

katharosada Jan 21, 2025

katharosada commented Jan 22, 2025

Add docker to all components. #1

Add docker to all components. #1

Conversation

sennav commented Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sennav Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

katharosada commented Jan 21, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

katharosada commented Jan 22, 2025

sennav commented Jan 21, 2025 •

edited

Loading

sennav Jan 21, 2025 •

edited

Loading