[FR] packages.txt in a similar way to plugins.txt

[Petrox]

Please verify that this feature request has NOT been suggested before.

• I checked and didn't find a similar feature request

Problem statement

Some plugins require extra packages installed in the OS.
(eg wirewiz depends on graphviz, which means local customization of the stable docker images)

Suggested solution

It could be done that a file containing the required extra packages could be stored together with the plugins.txt and any packages (at specific versions) would be installed via apt-get before the packages.

Describe alternatives you've considered

Note: while graphviz can be installed via pip (and thus could be in a requirements.txt for the plugin which could be installed by inventree at startup, the generic idea of being able to add common tools (image or document or media conversion from ffmpeg to imagemagick) would open easily accessible possibilities. (And since those packages should be secure-by-default, in theory this should not open attack surfaces)

Examples of other systems

plugins.txt?

Do you want to develop this?

• I want to develop this.

[SchrodingersGat]

@Petrox would be a potential addition, although we would need to carefully consider the security implications.

Edit: (I was trying to find the previous discussion as mentioned by @matmair below). This would be for a very advanced use case.

[matmair]

We had long discussions between core devs about this already (multiple times), this makes the docker images unstable and hard to debug.

[Petrox]

Ok, agreed it makes them more fragile, but I see not much difference than installing pip packages.

Just for reference:

graphviz is available through pip so it could be set as wireviz plugin dep, installed automatically by pip
inventree/inventree-wireviz#37

[Petrox]

@Petrox would be a potential addition, although we would need to carefully consider the security implications.

From security viewpoint I think it is much easier to distribute malicious pip packages than malicious ubuntu signed packages, but both lists are managed by the admin, so if he installs something that breaks the container, he can just redo it without the broken package.

[Petrox]

Edit: (I was trying to find the previous discussion as mentioned by @matmair below). This would be for a very advanced use case.

I don't think trying to use the wirewiz plugin as it is written in the readme is considered a very advanced use case.

IMHO To have happy endusers plugins should work smoothly out of the box (limited by security ofc, but no containerimage-and-database-restore-process-manipulation-magic should be necessary)

[matmair]

@Petrox I am happy to review a solution with full test coverage and a stable set of e2e tests.
The burden on a hobby user to execute the necessary steps is small, everything else is probably an enterprise use case. I do not intend to spend time on them for free in my spare time. You are welcome to tough.

[wolflu05]

Related to #3562, but this is not the only discussion about this, I think there is a continuation somewhere else.

[Petrox]

Proposed solution to solve the issue with much less effort on dev side: Include a few paragraphs (or page) in the doc about how to prep a customized container. I might contribute if it's an acceptable resolution.

…

On Tue, Oct 17, 2023 at 7:54 AM Matthias Mair ***@***.***> wrote: @Petrox <https://github.com/Petrox> I am happy to review a solution with full test coverage and a stable set of e2e tests. The burden on a hobby user to execute the necessary steps is small, everything else is probably an enterprise use case. I do not intend to spend time on them for free in my spare time. You are welcome to tough. — Reply to this email directly, view it on GitHub <#5718 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFKXLIKTXCJCKXBNVEIG3X7YMRRAVCNFSM6AAAAAA6DAD4JGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVG4YTMMBYGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[matmair]

That would certainly be a faster way if done right.

[github-actions]

This issue seems stale. Please react to show this is still important.

[wolflu05]

would still be good to have some docs for that, because we/I get questions from time to time about how to do that (latest: wolflu05/inventree-cups-plugin#5)

[SchrodingersGat]

I'm happy to review an update to the docs if someone wants to tackle it. The potential issue is that we intentionally purge a lot of build dependencies to keep the container size small, so it may be difficult to install extra packages depending on what they are

[Petrox]

Let's label this not just "docker" docs, but something like "Operational best practices" It could include tips and tricks for ops and admins, including backups, database syncs, docker modding, etc. But first I will try to write something here and then we can move forward based o your feedback.

…

On Sat, Dec 23, 2023, 14:44 Oliver ***@***.***> wrote: I'm happy to review an update to the docs if someone wants to tackle it. The potential issue is that we intentionally purge a lot of build dependencies to keep the container size small, so it may be difficult to install extra packages depending on what they are — Reply to this email directly, view it on GitHub <#5718 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFKXKJ2ACOXDF3XUCA26DYK3N47AVCNFSM6AAAAAA6DAD4JGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGI4TONZQHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Petrox]

I have a working setup with a few simple scripts, so I just need to explain it in a few paragraphs.

…

On Sat, Dec 23, 2023, 14:42 Lukas ***@***.***> wrote: would still be good to have some docs for that, because we/I get questions from time to time about how to do that (latest: wolflu05/inventree-cups-plugin#5 <wolflu05/inventree-cups-plugin#5>) — Reply to this email directly, view it on GitHub <#5718 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFKXIAAIS2WY35CJIM7LTYK3NU3AVCNFSM6AAAAAA6DAD4JGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGI4TOMZVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[github-actions]

This issue seems stale. Please react to show this is still important.

[SchrodingersGat]

not stale. @Petrox where are you at with this?

[Petrox]

Thx for pinging, I'll create some dockerfiles and a brief description to help new users "mod" their images easily.

[github-actions]

This issue seems stale. Please react to show this is still important.

[Petrox]

I will merge it info https://docs.inventree.org/en/latest/start/docker_install/ after approval, but it's probably easier to just chat about the content here.

Feel free to comment.

Inventree Container Customization

In some cases changing the official docker images might be necessary for additional features. For example the WireWiz Plugin depends on graphviz which is not installed in the official docker images.

To solve this problem we will create our own inventree images based on a specific inventree image, store it in the local docker image store, and then use it with the normal docker compose instead of the original image.

You already have a folder with the .env, docker-compose.yml files, so that would be a place for our own Dockerfile:

ARG sourcetag=stable
FROM inventree/inventree:$sourcetag
ARG packagelist
ENV packages $packagelist
RUN apt-get update && (for pkg in $packages ; do echo "Installing apt-get install $pkg"; apt-get install $pkg -y ; done) || (apk update ; for pkg in $packages ; do echo "Installing apk add $pkg"; apk add $pkg ; done )

To make it more flexible, this Dockerfile needs two parameters: sourcetag a tag from docker hub and a list of packages to install.

Note: while the new images are minimal alpine based images the older inventree versions were built on debian based images, and there might be situations where it is beneficial, so the above script tries to install the packages with apt-get and then with apk (the alpine package manager).

To build our image with graphviz and lsof extra packages:

sourcetag="0.14.3"
docker build . --build-arg sourcetag=$sourcetag --build-arg "packagelist=graphviz lsof" -t inventree/inventree:custom-$sourcetag -t inventree/inventree:custom

Then edit your .env file:

INVENTREE_TAG=custom-0.14.5

Note: we don't use docker push since we don't want to upload our modded images anywhere right now (and luckily we are not authorized to push an image to the official repo), so it will stay just in the docker hosts image file cache. It is not an issue, since we can recreate the images should we have to.

Note: it's best to use a specific version rather than "stable", since then you will build 0.14.3, 0.14.4, 0.14.5 images independently and can upgrade and maybe roll back to previous versions.

Upgrading versions quickly

A good practice is to have a staging environment running on the same host (with just a little change in the .env file) and the staging system could be updated with the production database, upgraded to any new version and when it is proven to work properly the production system can be upgraded nearly instantly since the image is already there.

Of course maintaining a production system means you must make backups often and store it remotely.

If you plan to switch inventree versions quickly, you might want to use a version independent (inventree/inventree:custom) tag in your .env file:

INVENTREE_TAG=custom

Then you might run this to upgrade to any specific version:

sourcetag=0.14.5
docker compose down
docker build . --build-arg sourcetag=$sourcetag --build-arg "packagelist=graphviz mc lsof" -t inventree/inventree:custom-$sourcetag -t inventree/inventree:custom
docker compose run inventree-server invoke migrate
docker compose down
docker compose up

[github-actions]

This issue seems stale. Please react to show this is still important.