Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC Request: http auditing / recording / caching #46

Open
dustmop opened this issue Jul 8, 2019 · 0 comments
Open

RFC Request: http auditing / recording / caching #46

dustmop opened this issue Jul 8, 2019 · 0 comments

Comments

@dustmop
Copy link
Contributor

dustmop commented Jul 8, 2019

We should have some mechanism for recording http requests and responses in starlark. There's a few levels at which this could work:

  1. when developing a transform script, cache an http response and replay it on future runs, so that the destination site doesn't get hammered, and also doesn't send back different data that breaks in-progress code
  2. record all http responses so that we maintain an audit trail and make transforms be fully reproducible
  3. only record http responses as hashes or checksums, so we can at least tell users when something has changed. May need some kind of smart differ that ignores unimportant dynamic content

When caching http responses, we have a few ways to handle them:

  1. Record the response, and freeze it during development, never hitting the server again
  2. Set the timestamp when an http request is made, send the http header "If-Modified-Since", and upon receiving a 304 status code, supply the cached version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant