Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

serve static files from CDN #41

Closed
hnykda opened this issue Mar 13, 2020 · 8 comments
Closed

serve static files from CDN #41

hnykda opened this issue Mar 13, 2020 · 8 comments
Assignees
Labels

Comments

@hnykda
Copy link
Contributor

hnykda commented Mar 13, 2020

Only if it's actually needed. We could reduce some load on the server if we hosted the static files elsewhere (JS and CSS). That would reduce responsivness of the site (userfriendlines) and reduce cost on compute cost and egress between the client and the server.

https://epidemicforecasting.slack.com/archives/C0100EK7N8N/p1584518497196200?thread_ts=1584472523.152600&cid=C0100EK7N8N

Solution:

  • add gsutil cp -Z -a <somecacheheaders as in epifor> ... step which would take the content of src/server/static and upload it to the bucket (for staging and production independently), ideally two way sync
  • setup the static_url to be served from GCS in deployments, maintain the file serving locally, that likely means dealing with url_for , maybe https://fastapi.tiangolo.com/tutorial/static-files/ and changing url_for from /static to something like https://storage.googleapis.com/static-covid/static/
@hnykda hnykda added the devops label Mar 13, 2020
@gavento
Copy link
Contributor

gavento commented Mar 24, 2020

Note: Main data files (JSON) are already served from GCS storage, which scales extremely well. We can upload assets too, I guess?

@jez4
Copy link

jez4 commented Mar 30, 2020

@hnykda I am checking the deploy action in pythonapp.yml. I should add the gsutil cp there? I will need some more description :)

@hnykda
Copy link
Contributor Author

hnykda commented Mar 30, 2020

Yes, it should be there on a deploy step. The command is IMO equivalent to the `push_to_bucket

The hard part seems to be actually rendering templates with different static URL paths for static files (I asked about it here encode/starlette#884).

In the first case, I propose to use something like:

APP_DEBUG = True  # local dev
STATIC_URL= "/static" if APP_DEBUG else "https://storage.googleapis.com/static-covid/static/"

...
def someview():
    return Template("<template>.html", {STATIC_URL: STATIC_URL})  # using e.g. `{{STATIC_URL}}/js/lines.js`

@hnykda
Copy link
Contributor Author

hnykda commented Mar 31, 2020

@jez4 I just realized that I did something in the past (it's not in master anymore): https://github.com/epidemics/covid/pull/122/files

^ you IMO want to take this and move it to the deploy branch only (so it's iterated only on deploy) and in addition of setting that public access and cache control actually uploading everything gzipped.

@jez4
Copy link

jez4 commented Apr 1, 2020

I should use {{ STATIC_URL }} for templates

@jez4
Copy link

jez4 commented Apr 1, 2020

I have managed to upload static files to the bucket gzipped. Now the templates.

@jez4
Copy link

jez4 commented Apr 1, 2020

The templates are done. Now just to perfect the workflow. I will ask for review afterwards.

@hnykda
Copy link
Contributor Author

hnykda commented Apr 1, 2020

Closed via #333

@hnykda hnykda closed this as completed Apr 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants