dependency needs to be added to setup.cfg too

[charlienegri]

Summary: #233 was missing a dependency in setup.cfg

Related issue: #220

Suggested reviewer(s):

Reviewer checklist:

• The headers of all files contain a reference to the repository license
• 100% test coverage of new code - meaning:
  • The overall test coverage increased or remained the same as before
  • Every function is accompanied with a test suite
  • Tests are both positive (testing that the function work as intended with valid data) and negative (testing that the function behaves as expected with invalid data, e.g., that correct exceptions are thrown)
  • Functions with optional arguments have separate tests for all options
• Examples are supported by doctests
• All tests are passing
• All names (e.g., files, classes, functions, variables) are explicit
• Documentation (as docstrings) is complete and understandable

The checklist is based on the S-ENDA conventions and definition of done (see General Conventions). The above points are not necessarily relevant to all contributions. In that case, please add a short explanation to help the reviewer.

[magnarem]

approved

[magnarem]

approved and merged

[charlienegri]

this setup works for the local test but I see no /metrics endpoint exposed now in dev and I wonder if the real case of having gunicorn multiprocesses might require something more https://github.com/rycus86/prometheus_flask_exporter?tab=readme-ov-file#wsgi, will try to investigate

[magnarem]

Yes. You are right that you probably have to take in account that dmci is running in gunicorn with multiple processes, so you get the metrics from all processes.

But I see another problem in dmci code, that makes the /metrics endpoint not working on k8s, and working locally.

When you test dmci locally from the dmci_start_api.py then the prometheus_flask_exporter defined in api_main() (https://github.com/metno/discovery-metadata-catalog-ingestor/blob/main/dmci/__init__.py) is loaded.

But when dmci is run in the Docker container, the initializing of the app is happening in container/wsgi.py, and this code does not execute the api_main() function.
So in this wsgi.py, you will probably need to add the prometheus_flask_exporter as well, for it to work with wsgi

And for testing with gunicorn, build the container locally and test it locally with wsgi gunicorn (The default entry point in the container uses the gunicorn)

[charlienegri]

right, thanks for pointing that out, I had not realized