This is the official back-end service of PolicyEngine, a non-profit with the mission of computing the impact of public policy for the world.
Running or editing the API locally will require a Python virtual environment, either through the Python venv command or a secondary package like conda. For more information on how to do this, check out the documentation for venv here and this overview blog post for conda here.
Python 3.10 or 3.11 is required.
All of our code changes are made against a GitHub issue. If you're new to the project, go to Issues and search for good first issues label: "good first issue".
Currently, we don't typically assign contributors. If you see an open issue that no one's opened a PR against, it's all yours! Feel free to make some edits, then open a PR, as described below.
git clone https://github.com/PolicyEngine/policyengine-api.git
To contribute, clone the repository instead of forking it and then request to be added as a contributor. Create a new branch and get started!
make install
Run:
make debug
Now you're ready to start developing!
NOTE: If you are using Airpods or other Apple bluetooth products, you may get an error related to the port. If this is the case, define a specific port in the debug statement in the Makefile. For example:
debug:
FLASK_APP=policyengine_api.api FLASK_DEBUG=1 flask run --without-threads --port=5001
If you get a CORS error try:
In api.py, comment out
CORS(app)
Add
CORS(app, resources={r"/*": {"origins": "*"}})
A simple API get call you can send in Postman to make sure it is working as expected is:
http://127.0.0.1:5001/us/policy/2
- In policyengine-app/src/api/call.js, comment out
const POLICYENGINE_API = "https://api.policyengine.org";
And add
const POLICYENGINE_API = "http://127.0.0.1:5001" (or the relevant port where the server is running)
- Start server as described above
- Start app as described in policyengine-app/README.md
NOTE: Any output that needs to be calculated will not work. Therefore, only household output can be tested with this setup.
To test anything that utilizes Redis or the API's service workers (e.g. anything that requires society-wide calculations with the policy calculator), you'll also need to complete the following steps:
- Start Redis
- Install Redis:
brew install redis
- Start Redis:
redis-server
- Start the API service worker
- This requires getting the POLICYENGINE_DB_PASSWORD from a teammate
- A teammate also must add you as a Google Cloud user and you will need to configure Application Default Credentials using your Google Account. Then complete the following steps:
-
Install Google Cloud CLI as described in the ADC documentation
-
Run the following to initialize it
gcloud init
When prompted "Pick cloud project to use:" Select the number that corresponds to "policyengine-api" When prompted "Do you want to configure a default Compute Region and Zone? (Y/n)?" Select 'n'
Create the credentials:
gcloud auth application-default login
Return to policyengine-api folder and relevant python environment and run:
POLICYENGINE_DB_PASSWORD="PASSWORD_HERE" python policyengine_api/worker.py
NOTE: Calculations are not possible in the uk app without access to a specific dataset. Expect an error: "ValueError: Invalid response code 404 for url https://api.github.com/repos/policyengine/non-public-microdata/releases/tags/uk-2024-march-efo."
You've finished your contribution, but now what? Before opening a PR, we ask contributors to do three things.
To test your changes against our series of automated tests, run
make debug-test
NOTE: Running the command make test will fail, as this command is utilized by the deployed app to run tests and requires passwords to the production database.
We also ask that you add tests for any new features or bug-fixes you add, so we can gradually build up the code coverage. Our tests are written in the Python standard, Pytest, and will be run again against the production environment, as well.
In addition to the tests, we use Black to lint our codebase, so before opening a pull request, Step 2 is to lint the code by running
make format
This will automatically format the code for you; no need to do anything else.
Finally, we ask contributors to make it clear for our users what changes have been made by contributing to a changelog. This changelog is formatted in YAML and describes the changes you've made to the code. This should follow the below format:
- bump: {major, minor, patch}
changes:
{added, removed, changed, fixed}:
- <variable or program>
For more info on the syntax, check out the semantic versioning docs and keep a changelog.
Write your changelog info into the empty file called changelog_entry.yaml. When you open your PR, this will automatically be added to the overall changelog.
Now you've finished your contribution! Please open a pull request (PR) from your branch and request review. At times, it may take some time for the team to review your PR, especially for larger contributions, so please be patient--we will be sure to get to it.
In the first line of your PR, please make sure to include the following:
Fixes {issue_number}
This makes it much easier for us to maintain and prune our issue board.
Please try to be detailed in your PRs about the changes you made and why you made them. You may find yourself looking back at them for reference in the future, or needing insight about someone else's changes. Save yourself a conversation and write it all in the PR!
Here are some best practices for using Git.
When you're ready for review, switch the PR from Draft to Ready for review and add a contributor as a reviewer.
Distributed under the AGPL License. See LICENSE for more info.
Thanks to Othneil Drew for his README template.
