Add 'docker-compose' file #1163

toddgardner · 2024-04-05T03:28:32Z

This allows a run of a local development platform with 'docker-compose up' that runs both the database and server. The only tricky implementation is relying on the health checks as mysql takes a bit of time to startup.

chaoticbear

No blockers from me if this actually is working. I just made a few notes on how my docker-compose setups are usually set up. Also we should probably add some documentation to the README

docker-compose.yml

chaoticbear

A couple more quick suggestions. Also I tried to spin it up and couldn't log in. I'm not sure why. I can't remember if there's a command you need to run to seed the users from conf/directory.yaml or if it's supposed to just work.

docker-compose.yml

toddgardner · 2024-04-06T01:55:13Z

A couple more quick suggestions. Also I tried to spin it up and couldn't log in. I'm not sure why. I can't remember if there's a command you need to run to seed the users from conf/directory.yaml or if it's supposed to just work.

Sorry mentioned in my slack, I was thinking of seeding a database but I think I was just overcomplicating it; the current pushed version just uses the DMS file backend,and should work from the instructions in the repo:

    cp conf/directory-sample.yaml conf/directory.yaml

    docker-compose up

I've confirmed at least locally I can log in as Parenthetical.

chaoticbear · 2024-04-06T02:40:39Z

A couple more quick suggestions. Also I tried to spin it up and couldn't log in. I'm not sure why. I can't remember if there's a command you need to run to seed the users from conf/directory.yaml or if it's supposed to just work.

Sorry mentioned in my slack, I was thinking of seeding a database but I think I was just overcomplicating it; the current pushed version just uses the DMS file backend,and should work from the instructions in the repo:
    cp conf/directory-sample.yaml conf/directory.yaml

    docker-compose up
I've confirmed at least locally I can log in as Parenthetical.

@toddgardner don't you also need to copy the conf/imsd.conf and adjust it for the docker DB settings?

toddgardner · 2024-04-06T03:21:08Z

A couple more quick suggestions. Also I tried to spin it up and couldn't log in. I'm not sure why. I can't remember if there's a command you need to run to seed the users from conf/directory.yaml or if it's supposed to just work.

Sorry mentioned in my slack, I was thinking of seeding a database but I think I was just overcomplicating it; the current pushed version just uses the DMS file backend,and should work from the instructions in the repo:
    cp conf/directory-sample.yaml conf/directory.yaml

    docker-compose up
I've confirmed at least locally I can log in as Parenthetical.
@toddgardner don't you also need to copy the conf/imsd.conf and adjust it for the docker DB settings?

In the previous version when the above comment was written it was just relying on ENV vars in the docker-compose. In the one I just pushed I overwrote CMD in the docker-compose so I can pass the config file (and moved all settings into the imsd.conf that could be moved there, which is everything except IMS_DIRECTORY)

chaoticbear · 2024-04-06T04:33:42Z

A couple more quick suggestions. Also I tried to spin it up and couldn't log in. I'm not sure why. I can't remember if there's a command you need to run to seed the users from conf/directory.yaml or if it's supposed to just work.

Sorry mentioned in my slack, I was thinking of seeding a database but I think I was just overcomplicating it; the current pushed version just uses the DMS file backend,and should work from the instructions in the repo:
    cp conf/directory-sample.yaml conf/directory.yaml

    docker-compose up
I've confirmed at least locally I can log in as Parenthetical.
@toddgardner don't you also need to copy the conf/imsd.conf and adjust it for the docker DB settings?
In the previous version when the above comment was written it was just relying on ENV vars in the docker-compose. In the one I just pushed I overwrote CMD in the docker-compose so I can pass the config file (and moved all settings into the imsd.conf that could be moved there, which is everything except IMS_DIRECTORY)

@toddgardner I didn't realize we could get away with not using the imsd.conf and just use env vars. For development at least I think that's better because then we can put all of our options in one .env file.

I re-based my PR on yours, take a look and if you don't hate it, go ahead and merge it into yours https://github.com/toddgardner/ranger-ims-server/pull/1

I added overrides for all of the env vars so that when we're tweaking things we can just adjust our local .env and not end up committing changes to the docker-compose. I also added networking which corresponds to the PR I have started on the web client burningmantech/ranger-ims-web#1038

chaoticbear

@toddgardner This is pretty close to merge-able in my opinion. Thanks for your work on this and for dealing with my nit-picks.

We have a couple linter errors. I think the sdist error would be solved by adding the following lines on: https://github.com/burningmantech/ranger-ims-server/blob/master/MANIFEST.in

include conf/imsd-docker-compose-sample.conf
...
exclude docker-compose.yml
recursive-exclude .docker

@wsanchez let me know if you agree about that fix. I am not that familiar with the sdist setup. And if you have an idea about the other linter error: https://github.com/burningmantech/ranger-ims-server/actions/runs/8581948224/job/23520484140?pr=1163

Dockerfile

chaoticbear · 2024-04-08T16:52:10Z

MANIFEST.in

@@ -24,7 +25,9 @@ include pyproject.toml
 include README.rst
 include requirements/requirements*.txt
 include tox.ini
+recursive-include conf *.conf


The sdist should only include source files, not configs used for dev testing, I think.

chaoticbear

LGTM

@wsanchez wanna take a look at this before it gets merged?

My impression is that the remaining CI issue is the same as this issue #1156 and we can ignore it for this PR

chaoticbear · 2024-04-11T20:45:07Z

LGTM

@wsanchez wanna take a look at this before it gets merged?

My impression is that the remaining CI issue is the same as this issue #1156 and we can ignore it for this PR

@toddgardner can you re-base with the latest from master

… wtih 'docker-compose up'

Co-authored-by: Bear <[email protected]>

Support faster development environment provisioning with a docker & docker-compose setup.

Supports external ranger-ims-web FE project with access to the backend API.

for more information, see https://pre-commit.ci

toddgardner · 2024-04-12T17:57:57Z

LGTM
@wsanchez wanna take a look at this before it gets merged?
My impression is that the remaining CI issue is the same as this issue #1156 and we can ignore it for this PR

@toddgardner can you re-base with the latest from master

Done

wsanchez · 2024-04-22T16:25:13Z

MANIFEST.in

@@ -1,3 +1,4 @@
+exclude docker-compose.yml


Why exclude? It's part of the source repo…

@wsanchez I think we're unclear on what should be included. Is this a production distribution package? If so the docker-compose isn't really set up for production just yet, but could be.

No… Manifest.in is only used for the source distribution.

wsanchez · 2024-04-22T16:44:27Z

.gitignore

+*.env
+/.docker/*
+!/.docker/sample.env


What reads these *.env files? docker-compose?

Yes, docker-compose reads them in and uses them to populate vars in the file, some of which are passed through to the services.

wsanchez · 2024-04-22T16:52:25Z

README.rst

+
+    # First time setup; these files are mounted into the container
+    cp conf/imsd-docker-compose-sample.conf conf/imsd.conf
+    cp conf/directory-sample.yaml conf/directory.yaml


If we are starting MySQL up we probably should also use that for the directory, but it would be a pain to set up from scratch.

We need a pair of dump files to add in this process: one for a starting directory and another for a IMS DB with something to fiddle with.

I guess that could be a follow-on PR; no need to block here.

codecov · 2024-04-23T22:51:42Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.05%. Comparing base (774610e) to head (8c62207).
Report is 1 commits behind head on master.

❗ Current head 8c62207 differs from pull request most recent head ee4c415. Consider uploading reports for the commit ee4c415 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1163      +/-   ##
==========================================
+ Coverage   69.03%   69.05%   +0.02%     
==========================================
  Files         187      187              
  Lines        9051     9051              
  Branches     1484     1484              
==========================================
+ Hits         6248     6250       +2     
+ Misses       2702     2701       -1     
+ Partials      101      100       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

chaoticbear approved these changes Apr 5, 2024

View reviewed changes

docker-compose.yml Outdated Show resolved Hide resolved

docker-compose.yml Outdated Show resolved Hide resolved

docker-compose.yml Outdated Show resolved Hide resolved

chaoticbear reviewed Apr 5, 2024

View reviewed changes

docker-compose.yml Outdated Show resolved Hide resolved

docker-compose.yml Show resolved Hide resolved

chaoticbear requested changes Apr 6, 2024

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

chaoticbear mentioned this pull request Apr 6, 2024

[FEATURE] Create docker-compose config #1166

Open

7 tasks

chaoticbear reviewed Apr 8, 2024

View reviewed changes

chaoticbear self-requested a review April 11, 2024 20:43

chaoticbear approved these changes Apr 11, 2024

View reviewed changes

toddgardner and others added 18 commits April 12, 2024 13:56

Add a simple 'docker-compose' file to run the server in a single step…

4a27ebd

… wtih 'docker-compose up'

Update docker-compose.yml

b7d5afa

Co-authored-by: Bear <[email protected]>

Add the options file to docker compose

8b38684

Add docker compose networking & environment opts

0c85efc

Support faster development environment provisioning with a docker & docker-compose setup.

Add shared external docker network

b6e6dca

Supports external ranger-ims-web FE project with access to the backend API.

Standardize service & container names

93b4d00

Adjust docker network config

d1897fd

Add Docker dev steps to README

a1fe738

Remove unnecessary Dockerfile

5324b1c

Add env var overrides to docker compose config

866679b

Move sample.env - .docker & adjust gitignore

1e52558

Add db image override option

8565afb

Consolidate DC instructions, add env var info

0e4a95a

Remove extra imsd sample conf

c8bb71e

Fix db image syntax

4f10ba4

Match db password from sample conf

c7b6c35

Add MARIADB_USER to set username

9462586

[pre-commit.ci] auto fixes from pre-commit.com hooks

4a5fc13

for more information, see https://pre-commit.ci

toddgardner added 4 commits April 12, 2024 13:56

Fix overlines

59de52b

Add items to manifest, remove test code

649af9a

Adjust manifest

1b8a9ef

Adjust MANIFEST to make lint pass

212593a

wsanchez reviewed Apr 22, 2024

View reviewed changes

Merge branch 'master' into todd/docker-compose

1787049

wsanchez reviewed Apr 22, 2024

View reviewed changes

Update MANIFEST.in with sdist in mind.

8c62207

chaoticbear mentioned this pull request Apr 23, 2024

[INTERNAL] Prep DB dumps for dev setup #1183

Open

Merge branch 'master' into todd/docker-compose

ee4c415

wsanchez enabled auto-merge April 23, 2024 22:51

wsanchez merged commit 6f11f13 into burningmantech:master Apr 23, 2024
11 of 12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add 'docker-compose' file #1163

Add 'docker-compose' file #1163

toddgardner commented Apr 5, 2024

chaoticbear left a comment

chaoticbear left a comment

toddgardner commented Apr 6, 2024

chaoticbear commented Apr 6, 2024

toddgardner commented Apr 6, 2024

chaoticbear commented Apr 6, 2024 •

edited

Loading

chaoticbear left a comment •

edited

Loading

chaoticbear Apr 8, 2024

wsanchez Apr 22, 2024

chaoticbear left a comment

chaoticbear commented Apr 11, 2024

toddgardner commented Apr 12, 2024

wsanchez Apr 22, 2024

chaoticbear Apr 22, 2024

wsanchez Apr 22, 2024

wsanchez Apr 22, 2024

chaoticbear Apr 22, 2024

wsanchez Apr 22, 2024

codecov bot commented Apr 23, 2024

Add 'docker-compose' file #1163

Add 'docker-compose' file #1163

Conversation

toddgardner commented Apr 5, 2024

chaoticbear left a comment

Choose a reason for hiding this comment

chaoticbear left a comment

Choose a reason for hiding this comment

toddgardner commented Apr 6, 2024

chaoticbear commented Apr 6, 2024

toddgardner commented Apr 6, 2024

chaoticbear commented Apr 6, 2024 • edited Loading

chaoticbear left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chaoticbear left a comment

Choose a reason for hiding this comment

chaoticbear commented Apr 11, 2024

toddgardner commented Apr 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Apr 23, 2024

Codecov Report

chaoticbear commented Apr 6, 2024 •

edited

Loading

chaoticbear left a comment •

edited

Loading