diff --git a/README.md b/README.md index 83dc04f..f660899 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ -## CommCareHQ Superset Integration +## CommCare HQ Superset Integration -This is a python package that can be installed alongside of `apache-superset` to integrate Superset and CommCareHQ. +This is a Python package that can be installed alongside of +`apache-superset` to integrate Superset and CommCare HQ. ## Local Development @@ -8,33 +9,75 @@ Follow below instructions. ### Setup env -While doing development on top of this integration, it's useful to install this via `pip -e` option so that any changes made get reflected directly without another `pip install`. +While doing development on top of this integration, it's useful to +install this via `pip -e` option so that any changes made get reflected +directly without another `pip install`. -- Setup a virtual environment. -- Clone this repo and cd into the directory of this repo. -- Run `cp superset_config.example.py superset_config.py` and override the config appropriately. +- Set up a virtual environment. +- Clone this repo and change into the directory of this repo. +- Run `cp superset_config.example.py superset_config.py` and override + the config appropriately. - Run `pip install -e .` -### CommCareHQ OAuth Integration +### CommCare HQ OAuth Integration -- Create an OAuth application on CommCareHQ using Django Admin `/admin/oauth2_provider/application/`. Use `/oauth-authorized/commcare` as the redirect URL. -- Update `OAUTH_PROVIDERS` setting in `superset_config.py` with OAuth client credentials obtained from HQ. +- Create an OAuth application on CommCare HQ using Django Admin + `/admin/oauth2_provider/application/`. Use + `/oauth-authorized/commcare` as the redirect URL. +- Update `OAUTH_PROVIDERS` setting in `superset_config.py` with OAuth + client credentials obtained from HQ. ### Initialize Superset -Run through the initialization instructions at https://superset.apache.org/docs/installation/installing-superset-from-scratch/#installing-and-initializing-superset. You may skip `superset load_examples`. - -You should now be able to run superset using the `superset run` command from the above docs. However OAuth login does not work yet as hq-superset needs a postgres database created to store CommCare HQ data. +Read through the initialization instructions at +https://superset.apache.org/docs/installation/installing-superset-from-scratch/#installing-and-initializing-superset. + +Create the database. These instructions assume that PostgreSQL is +running on localhost, and that its user is "commcarehq". Adapt +accordingly: +```bash +$ createdb -h localhost -p 5432 -U commcarehq superset_meta +``` + +Set the following environment variables: +```bash +$ export FLASK_APP=superset +$ export SUPERSET_CONFIG_PATH=/path/to/superset_config.py +``` + +Initialize the database. Create an administrator. Create default roles +and permissions: +```bash +$ superset db upgrade +$ superset fab create-admin +$ superset load_examples # (Optional) +$ superset init +``` +You may skip `superset load_examples`, although they could be useful. + +You should now be able to run superset using the `superset run` command: +```bash +$ superset run -p 8088 --with-threads --reload --debugger +``` +However, OAuth login does not work yet as hq-superset needs a Postgres +database created to store CommCare HQ data. ### Create a Postgres Database Connection for storing HQ data -- Create a Postgres database -- Login to superset as the admin user created in the Superset installation and initialization. Note that you will need to update `AUTH_TYPE = AUTH_DB` to login as admin user. `AUTH_TYPE` should be otherwise set to `AUTH_OAUTH`. +- Create a Postgres database. e.g. + ```bash + $ createdb -h localhost -p 5432 -U commcarehq hq_data + ``` +- Log into Superset as the admin user created in the Superset + installation and initialization. Note that you will need to update + `AUTH_TYPE = AUTH_DB` to log in as admin user. `AUTH_TYPE` should be + otherwise set to `AUTH_OAUTH`. - Go to 'Data' -> 'Databases' or http://127.0.0.1:8088/databaseview/list/ -- Create a Database connection by clicking '+ DATABASE' button at the top. -- The name of the DISPLAY NAME should be 'HQ Data' exactly, as this is the name by which this codebase refers to the postgres DB. +- Create a database connection by clicking '+ DATABASE' button at the top. +- The name of the DISPLAY NAME should be 'HQ Data' exactly, as this is + the name by which this codebase refers to the Postgres DB. OAuth integration should now start working. @@ -44,19 +87,22 @@ OAuth integration should now start working. Celery is used to import UCRs that are larger than `hq_superset.views.ASYNC_DATASOURCE_IMPORT_LIMIT_IN_BYTES`. If you need -to import UCRs larger than this, you need to run celery to import them. +to import UCRs larger than this, you need to run Celery to import them. Here is how celery can be run locally. - Install and run Redis -- Add Redis and celery config sections from `superset_config.example.py` to your local `superset_config.py`. -- Run `celery --app=superset.tasks.celery_app:app worker --pool=prefork -O fair -c 4` in the superset virtualenv. +- Add Redis and Celery config sections from + `superset_config.example.py` to your local `superset_config.py`. +- Run + `celery --app=superset.tasks.celery_app:app worker --pool=prefork -O fair -c 4` + in the Superset virtualenv. ### Testing Tests use pytest, which is included in `requirements_dev.txt`: - $ pip install -r requirements_dev.txt + $ pip install -r requirements_test.txt $ pytest The test runner can only run tests that do not import from Superset. The