Adapter that pulls data from RapidPro and makes it available in PostgreSQL for ease of analysis.
PostgreSQL 9.6 and greater. The user passed in the postgres url needs to have full creation rights on the given database.
- Clone the repository
- Update environment variables and network mode in
docker-compose.yml
POSTGRESQL_URL
is the PostgreSQL urlRAPIDPRO_URL
is the url of your RapidPro deploymentRAPIDPRO_AUTH
is the RapidPro API Token, without the prefix
- Run
docker-compose up
- Run step 3 for subsequent updates
Currently, the adapter pulls, but not limited to the following data.
- Contacts
- Messages
- Runs
- Flows and definitions
- Runs migrations in
/migrations
to createrapidpro_contacts, rapidpro_messages, rapidpro_runs
tables and corresponding materialized viewsuseview_*
- Fetches data from RapidPro API on the respective endpoints and inserts to the tables.
- Each table has one column
doc
, similar tocouchdb
of couch2pg adapter, and one column with an id or uuid, uniquely identifying each row. - Refreshes the three materialized views
- RapidPro API token refreshes periodically, hence the need for fresh builds for each new token.
Run linting, unit and integration tests with:
docker-compose -f ./test/test-compose.yml up --build --abort-on-container-exit
Some environment variables can be set:
INT_PG_HOST
postgres host, defaults to postgresINT_PG_USER
postgres user, defaults to postgres. This user must be able to create databases on the given host.INT_PG_PASS
user's password, defaults to none (system default)INT_PG_DB
test database to use, defaults torapidpro2pgtest
INT_RP_PORT
RapidPro mocked server port, defaults to 6594
Use this command to tear down the containers.
docker-compose -f ./test/test-compose.yml down --volumes
The following RapidPro API end points are helpful to understand the tables and views created
- /api/v2/contacts - to list, create, update or delete contacts
- /api/v2/messages - to list messages
- /api/v2/runs - to list flow runs
- /api/v2/flows - to list flows
- /api/v2/definitions - to list flow definitions