[BUG] Error importing geodata when running container with a non-root user #5404

amitrea · 2023-11-30T08:30:01Z

The bug

After running the latest version of Immich (v1.89.0) and inspecting the logs of container immich-microservices I get the below error. I usually run containers with a non root user (uid:guid 1000:1000) for security reasons.
This error occurred only with this release.

❯ podman logs -f immich-microservices
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [NestFactory] Starting Nest application...
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +44ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] ScheduleModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] ConfigModule dependencies initialized +5ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +262ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] InfraModule dependencies initialized +5ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] DomainModule dependencies initialized +34ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [MetadataRepository] Initializing metadata repository
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [MetadataRepository] Importing geodata to database from file
[Nest] 3  - 11/30/2023, 9:00:54 AM   FATAL [MetadataRepository] Error importing geodata
[Nest] 3  - 11/30/2023, 9:00:54 AM   FATAL [MetadataRepository] Error: EACCES: permission denied, open '/usr/src/resources/admin1CodesASCII.txt'
[Nest] 3  - 11/30/2023, 9:00:54 AM   ERROR [MetadataService] Unable to initialize reverse geocoding: Error: EACCES: permission denied, open '/usr/src/resources/admin1CodesASCII.txt'
Error: EACCES: permission denied, open '/usr/src/resources/admin1CodesASCII.txt'
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [SearchService] Running bootstrap
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Schema up to date: assets/assets-v10
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Schema up to date: albums/albums-v2
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Schema up to date: faces/faces-v1
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Alias mapping: [{"collection_name":"faces-v1","name":"faces"},{"collection_name":"albums-v2","name":"albums"},{"collection_name":"assets-v10","name":"assets"}]
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Collections needing migration: {"assets":false,"albums":false,"faces":false}
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [NestApplication] Nest application successfully started +16ms
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [ImmichMicroservice] Immich Microservices is listening on http://[::1]:3002 [v1.89.0] [PRODUCTION]

The OS that Immich Server is running on

Fedora 38

Version of Immich Server

v1.89.0

Version of Immich Mobile App

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

...

  immich-microservices:
    image: ghcr.io/immich-app/immich-server:release
    container_name: immich-microservices
    command: ["./start-microservices.sh"]
    restart: unless-stopped
    user: 1000:1000
    group_add:
    - "105"
    env_file:
      - immich.env
    devices:
      - /dev/dri:/dev/dri:rw
    volumes:
      # - /data/immich/geocoding:/usr/src/app/.reverse-geocoding-dump:rw,z
      - /data/immich/upload:/usr/src/app/upload/upload:rw,z
      - /data/immich/library:/usr/src/app/upload/library:rw,z
      - /data/immich/thumbs:/usr/src/app/upload/thumbs:rw,z
      - /data/immich/profile:/usr/src/app/upload/profile:rw,z
      - /data/immich/encoded-video:/usr/src/app/upload/encoded-video:rw,z
      - /etc/localtime:/etc/localtime:ro,z
      - /data/certs/root_ca.crt:/etc/ssl/certs/root_ca.pem:ro,z
    depends_on:
      postgresqldb:
        condition: service_started
      redis:
        condition: service_started
      typesense:
        condition: service_started
      immich-server:
        condition: service_started

...

Your .env content

N/A

Reproduction steps

1) Deploy latest release (v1.89.0)
2) Inspected every Immich container logs
3) Catch the error above in immich-microservices container.

Additional information

I logged in immich-microservices container and checked the file permissions.

root@immich-microservices:/usr/src/app# ls -lah /usr/src/resources/
total 34M
drwxr-xr-x. 2 root root  102 Nov 25 03:05 .
drwxr-xr-x. 1 root root   17 Nov 25 03:27 ..
-rw-------. 1 root root 136K Nov 24 04:55 admin1CodesASCII.txt
-rw-------. 1 root root 2.2M Nov 24 04:55 admin2Codes.txt
-rw-r--r--. 1 root root  31M Nov 24 05:48 cities500.txt
-rw-r--r--. 1 root root   25 Nov 25 03:05 geodata-date.txt

It doesn't make sense to have the two files above admin1CodesASCII.txt and admin2Codes.txt non-readable by others.

One solution would be to change the permissions of the two files admin1CodesASCII.txt and admin2Codes.txt, in Dockerfile

The text was updated successfully, but these errors were encountered:

amitrea · 2023-11-30T08:44:25Z

After changing manually the two file permissions in immich-server (less important but for consistency) and immich-microservices the importing of geodata worked.

$ chmod 644 /usr/src/resources/*

Restart both containers immich-server, immich-microservices:

$ podman restart immich-server
$ podman restart immich-microservices
$ podman logs -f immich-microservices

...
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [NestFactory] Starting Nest application...
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +26ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] ConfigModule dependencies initialized +4ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +113ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] InfraModule dependencies initialized +2ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] DomainModule dependencies initialized +14ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [MetadataRepository] Initializing metadata repository
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [MetadataRepository] Importing geodata to database from file
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [MetadataRepository] Geodata import completed
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [MetadataService] Initialized local reverse geocoder
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [SearchService] Running bootstrap
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Schema up to date: assets/assets-v10
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Schema up to date: albums/albums-v2
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Schema up to date: faces/faces-v1
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Alias mapping: [{"collection_name":"faces-v1","name":"faces"},{"collection_name":"albums-v2","name":"albums"},{"collection_name":"assets-v10","name":"assets"}]
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Collections needing migration: {"assets":false,"albums":false,"faces":false}
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [NestApplication] Nest application successfully started +13ms
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [ImmichMicroservice] Immich Microservices is listening on http://[::1]:3002 [v1.89.0] [PRODUCTION]

bo0tzz · 2023-11-30T08:47:50Z

@zackpollard should we just run this chmod in the base image build?

amitrea · 2023-11-30T08:55:30Z

Yes. In "Additional information" section I pointed to the Dockerfile and line.

zackpollard · 2023-11-30T09:02:55Z

@zackpollard should we just run this chmod in the base image build?

Yes that makes sense, I'm not really sure why the file is created as 600 by docker at all tbh... Interesting default.

amitrea · 2023-11-30T09:06:33Z

Maybe there's a user mask somewhere which I cannot see.

I will try to provide a PR asap in the base server image repo.

baloo2 · 2023-11-30T11:22:43Z

I am experiencing exactly the same problem on my Synology. Stopped Immich project till fixed. Hopefully very soon...

otbutz · 2023-11-30T11:25:31Z

Workaround:

docker compose exec -it --user root immich-microservices bash
chmod 644 /usr/src/resources/*
docker compose restart immich-microservices

amitrea · 2023-11-30T11:26:35Z

It seems that Docker ADD URL_SRC DST run with umask 077.

I replaced all ADD commands + the RUN command related to geodata into one RUN command, using wget for downloading, with a small improvement on the size of dev base server image: from 2.31 GB to 2.30 GB. The prod base server image is 856 MB, which I think it is the same in both cases (with copy and wget).
The permissions on the files with this change are OK.

Please notice that the remote geodata files are dated with today, so probably they changed since the last release.

In several minutes I will come with the PR.

zackpollard · 2023-12-01T18:16:56Z

immich-app/base-images#21 should fix this, which will be included in the next Immich release. Please tag me if this persists after the next release has been published.

yodatak · 2023-12-01T23:45:57Z

Some command to run if k3s is use to fix permission

kubectl get pod  -n photos | grep micro
kubectl -n photos describe pod immich-microservices-b59d867bd-f2lln | grep 'Container ID:'
sudo k3s ctr task exec -t --exec-id myshell --user root YOURCONTENAIRIDHERE bash
chmod 644 /usr/src/resources/*

Nertskull · 2023-12-03T12:08:43Z

I had the same. And I tried the above work around with the chmod 644 which seems to have fixed the problem for the ASCII file.

But now I get the following error:

immich_microservices     | [Nest] 7  - 12/03/2023, 7:05:39 AM   ERROR [MetadataService] Unable to initialize reverse geocoding: Error: EACCES: permission denied, open '/usr/src/resources/geodata-date.txt'
immich_microservices     | Error: EACCES: permission denied, open '/usr/src/resources/geodata-date.txt'

I tried chmod with 777 and still the same error.

Ultimately this may not matter and may be fixed with the next release update. But just thought I'd document this. I'll check again once the next release is out.

amitrea · 2023-12-03T12:48:33Z

Log into the immich-microservices container with user root:
$ docker exec -it --user root immich-microservices bash

Run this:
$ ls -la /usr/src/resources/

Post the output you have.

The output should look like:

-rw-r--r--. 1 root root   138529 Nov 24 04:55 admin1CodesASCII.txt
-rw-r--r--. 1 root root  2251114 Nov 24 04:55 admin2Codes.txt
-rw-r--r--. 1 root root 32483181 Nov 24 05:48 cities500.txt
-rw-r--r--. 1 root root       25 Nov 25 03:05 geodata-date.txt

If the output is correct please try to restart immich-microservices container (this is neccessary if you change the files permissions):
$ docker restart immich-microservices

Check the logs for immich-microservices container:
$ docker logs -f immich-microservices

amitrea added bug labels Nov 30, 2023

amitrea mentioned this issue Nov 30, 2023

fix geodata files permissions immich-app/base-images#20

Closed

zackpollard closed this as completed Dec 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Error importing geodata when running container with a non-root user #5404

[BUG] Error importing geodata when running container with a non-root user #5404

amitrea commented Nov 30, 2023

amitrea commented Nov 30, 2023 •

edited

Loading

bo0tzz commented Nov 30, 2023

amitrea commented Nov 30, 2023

zackpollard commented Nov 30, 2023

amitrea commented Nov 30, 2023

baloo2 commented Nov 30, 2023

otbutz commented Nov 30, 2023

amitrea commented Nov 30, 2023

zackpollard commented Dec 1, 2023

yodatak commented Dec 1, 2023

Nertskull commented Dec 3, 2023

amitrea commented Dec 3, 2023

[BUG] Error importing geodata when running container with a non-root user #5404

[BUG] Error importing geodata when running container with a non-root user #5404

Comments

amitrea commented Nov 30, 2023

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Additional information

amitrea commented Nov 30, 2023 • edited Loading

bo0tzz commented Nov 30, 2023

amitrea commented Nov 30, 2023

zackpollard commented Nov 30, 2023

amitrea commented Nov 30, 2023

baloo2 commented Nov 30, 2023

otbutz commented Nov 30, 2023

amitrea commented Nov 30, 2023

zackpollard commented Dec 1, 2023

yodatak commented Dec 1, 2023

Nertskull commented Dec 3, 2023

amitrea commented Dec 3, 2023

amitrea commented Nov 30, 2023 •

edited

Loading