Migrated from navapbc#131
Originally created by @acouch on Tue, 02 Jul 2024 14:29:08 GMT

Summary

The current analytics readme and docs focuses on the previous version of the tool. It should be rewritten to reflect the recent changes.

It should highlight the fact that we have moved to capturing analytics in a postgres analytics database that will house our future analytics data and that we will use Metabase as a analyses and dashboard tool. It should highlight that system is architected so that the analytics work that will be part of the project can be run locally, though it is housed in AWS.

Acceptance criteria

• README.md should be updated.
  • The introduction should reflect the goals and structure noted above. Bonus points if it can link to the architecture diagram and if a separate diagram can be created to show the metabase -> analytics database <- ECS python service.
  • The directory structure should be updated or removed
  • An "Data Pipeline Service" section should be written that explains the python portion
  • A sub-section of "Data Pipeline" should include "Current Data" section that describes the current data and its structure
  • Using the tool should be renamed "Using the Slack Tool" or something, and a "Local Development" section be written with quick instructions to start the tool and run github report, ie 1. start the containers make start 1a. setup the github login or token 2. export the current datamake sprint-data-export && make issue-data-export 3. import the data make sprint-db-data-import and 4. view the data in metabase ie go to localhost:3100
• Technical Overview should be updated. The "Key Technologies" section could be updated to reflect that this is just the "Data pipeline service" part of the analytics. The "Orchestration" section should be be updated to include correct links https://github.com/navapbc/simpler-grants-gov/blob/main/documentation/analytics/technical-overview.md#orchestration
• development.md should be updated to focus on the steps that were created for the README.md . Create two sections, "Analytics Data Pipeline" and "Slack Integration Tool". The first section could link to the readme install steps or expand on them. Copy/paste the items from usage.md to here, but use the make commands instead of the local poetry. The Github token thing might need to be included. Setting up the tool locally could be moved under that latter setting so it is clear it is just for the slack integration piece. make setup doesn't exist AFIACT so should be removed.
• usage.md, should be updated so that the new stuff is removed and moved to 'development.md' and the file is renamed to "Usage of Slack Integration"