[Issue #2327] Configure API/backend to work with OpenSearch

[chouinar]

Summary

Fixes #2327

Time to review: 5 mins

Changes proposed

Modified configuration to call OpenSearch with our current auth setup

Adjusted the names of several env vars to match the Terraform setup

Adjusted the number of records we upsert per batch to OpenSearch from 5k to 1k

Context for reviewers

We will have another pass of auth setup later, getting proper auth setup with OpenSearch requires we adjust some other infra setup to get the auth in OpenSearch itself working, so right now we're logging in with the user/pass of the cluster itself. As those changes will take a bit to setup, I've just commented out the AWS auth approach we will use later.

Additional information

In dev I ran the load job + queried the API and it all functions. Queries go to the search index as expected and responses look good. Not sure about caching/performance, but was getting 100-200ms for the requests to the API.

For future reference, a decent way to query this in Logs Insights:

filter msg = 'end request'
| filter request.path = '/v1/opportunities/search'
| stats avg(response.time_ms), min(response.time_ms), max(response.time_ms) by bin(5m)

Here is the final big log statement from running the job in dev:

{
    "name": "src.task.task",
    "msg": "Completed %s in %s seconds",
    "args": [
        "LoadOpportunitiesToIndex",
        73.459
    ],
    "levelname": "INFO",
    "levelno": 20,
    "pathname": "/api/src/task/task.py",
    "filename": "task.py",
    "module": "task",
    "lineno": 48,
    "funcName": "run",
    "created": 1729014690.4964094,
    "msecs": 496,
    "relativeCreated": 80289.82591629028,
    "thread": 140513021029248,
    "threadName": "MainThread",
    "processName": "MainProcess",
    "process": 7,
    "taskName": null,
    "index_name": "opportunity-index-2024-10-15_13-50-17",
    "records_loaded": 16488,
    "task_duration_sec": 73.459,
    "app.name": "src.app",
    "environment": "dev",
    "task_name": "load-opportunity-data-opensearch",
    "task_uuid": "c237cc23-57e9-42b4-bb8d-3b69593cb470",
    "aws.ecs.task_name": "api-dev",
    "aws.ecs.task_id": "5d95f47fd9da4d359bf74134328e1947",
    "aws.ecs.task_definition": "api-dev:199",
    "aws.cloudwatch.log_group": "service/api-dev",
    "aws.cloudwatch.log_stream": "api-dev/api-dev/5d95f47fd9da4d359bf74134328e1947",
    "aws.step_function.id": null,
    "message": "Completed LoadOpportunitiesToIndex in 73.459 seconds"
}

[coilysiren]

It should be working now. Lemme think

[chouinar]

It should be working now. Lemme think

I just manually kicked it off for a first attempt.

[coilysiren]

I think that each individual commit that you want to deploy needs to include a change inside of api/*. Which is annoying, but I think that's what's missing.

[coilysiren]

Wow this was easy 💯

[coilysiren]

🚀