Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docker] A way mount the data directory that doesn't require chown #645

Open
bupahaibao opened this issue Jul 31, 2024 · 13 comments
Open

[docker] A way mount the data directory that doesn't require chown #645

bupahaibao opened this issue Jul 31, 2024 · 13 comments
Labels
help wanted Extra attention is needed

Comments

@bupahaibao
Copy link

I am using unraid. When I deployed the puter, I looked at the docker container log and found this error. Looking for technical help.
截屏2024-07-31 12 08 18

@KernelDeimos
Copy link
Contributor

KernelDeimos commented Jul 31, 2024

Hello, quick fix is to add NO_VAR_RUNTIME=1 before the command. The /var/puter directory is getting created and I still don't know why; this has been driving me nuts for weeks.

edit: I didn't read this carefully enough; didn't realize it was a Docker container.

This might be a permission issue. Can you provide the commands you used to reproduce this issue?

@bupahaibao
Copy link
Author

Hello, quick fix is to add NO_VAR_RUNTIME=1 before the command. The /var/puter directory is getting created and I still don't know why; this has been driving me nuts for weeks.

edit: I didn't read this carefully enough; didn't realize it was a Docker container.

This might be a permission issue. Can you provide the commands you used to reproduce this issue?


version: "3.8"
services:
puter:
container_name: puter
image: ghcr.io/heyputer/puter:latest
pull_policy: always
# build: ./
restart: unless-stopped
ports:
- '4100:4100'
environment:
TZ: Asia/Shanghai
CONFIG_PATH: /etc/puter
PUID: 1000
PGID: 1000
volumes:
- /mnt/nvme0n1-4/Configs/puter/config:/etc/puter
- /mnt/nvme0n1-4/Configs/puter/data:/var/puter
healthcheck:
test: wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
interval: 30s
timeout: 3s
retries: 3
start_period: 30s

@KernelDeimos
Copy link
Contributor

This is a docker compose file, right? Make sure you have ownership set correctly on those directories:

chown -R 1000:1000 /mnt/nvme0n1-4/Configs/puter/config
chown -R 1000:1000 /mnt/nvme0n1-4/Configs/puter/data

In case it helps, this is the snippet of code that's throwing the exception when checking /var/puter:

    require_write_permission: ({ path }) => {
        try {
            fs.writeFileSync(path_.join(path, '.tmp_test_write_permission'), 'test');
            fs.unlinkSync(path_.join(path, '.tmp_test_write_permission'));
        } catch (e) {
            throw new Error(`Cannot write to path: ${path}`);
        }
        return true;
    },

The function tries to write a file to that directory as a test. This error indicates that /var/puter is not writable within your container.

@KernelDeimos
Copy link
Contributor

@bupahaibao hello, did the solution above resolve the issue? If so, please let me know so I can close this, thanks!

@francescoferri
Copy link

Hello, I have the same situation on Ubuntu 22.04. Here is my compose:

services:
  puter:
    container_name: puter
    image: ghcr.io/heyputer/puter:latest
    pull_policy: always
    restart: unless-stopped
    ports:
      - 4100:4100
    environment:
      - PUID=1004
      - PGID=1004
      - TZ=Europe/Rome
    volumes:
      - /root/docker/stack/puter/config:/etc/puter
      - /root/docker/stack/puter/data:/var/puter
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 30s

I checked the user and group id were correct:

root@dockerstack:~/docker/stack# getent group 1004
container:x:1004:
root@dockerstack:~/docker/stack# id -u container
1004

It also looks like the permissions are correct:

root@dockerstack:~/docker/stack# ls -la puter/
total 16
drwxrwxr-x  4 container service 4096 Aug  9 11:14 .
drwxrwxr-x 13 container service 4096 Aug  9 11:13 ..
drwxrwxr-x  2 container service 4096 Aug  9 11:15 config
drwxrwxr-x  2 container service 4096 Aug  8 21:05 data

Could it be the permissions drwxrwxr-x?
I still get [BOOT/ERROR] Error: Cannot write to path: /var/puter and the container keeps restarting. Would love to get this awesome tool running!

@KernelDeimos
Copy link
Contributor

@francescoferri are you able to check the permissions of /var/puter inside the Docker container? I suspect that's where they're incorrect, but I'm not sure how Docker handles permissions of volume mounts.

@francescoferri
Copy link

So it looks like the container keeps restarting so I cannot make it into the shell:

root@dockerstack:~# docker exec -it puter sh
Error response from daemon: Container ec4a4f93b9013ec241cf51fafe7ed85afd52ad2b712bf847021aa2d371e118b3 is restarting, wait until the container is running

Because of this, I decided to start a webtop container with the same uid and gid just to see what is going on. Here is the compose:

webtop:
    image: lscr.io/linuxserver/webtop:latest
    container_name: webtop
    security_opt:
      - seccomp:unconfined #optional
    environment:
      - PUID=1004
      - PGID=1004
      - TZ=Europe/Rome
      - TITLE=Webtop #optional
    volumes:
      - /root/docker/stack/webtop/data:/config
      - /root/docker/stack/puter/config:/etc/puter
      - /root/docker/stack/puter/data:/var/puter
    ports:
      - 3000:3000
      - 3001:3001
    shm_size: "1gb" #optional
    restart: unless-stopped

Turns out that inside of the container at /var I see:

9ad4a2eb6d1f:/var$ ls -la
total 72
drwxr-xr-x 1 root root      4096 Aug 10 17:52 .
drwxr-xr-x 1 root root      4096 Aug 10 17:52 ..
drwxr-xr-x 1 root root      4096 Aug  3 20:53 cache
drwxr-xr-x 2 root root      4096 Aug  3 20:53 db
dr-xr-xr-x 2 root root      4096 Jul 27 15:31 empty
drwxr-xr-x 1 root root      4096 Aug  6 05:58 lib
drwxr-xr-x 2 root root      4096 Jul 27 15:31 local
drwxr-xr-x 3 root root      4096 Jul 27 15:31 lock
drwxr-xr-x 1 root root      4096 Aug  3 20:53 log
drwxr-xr-x 2 root root      4096 Jul 27 15:31 mail
drwxr-xr-x 2 root root      4096 Jul 27 15:31 opt
drwxrwxr-x 2 abc  kasm-user 4096 Aug 10 17:35 puter
lrwxrwxrwx 1 root root         4 Jul 27 15:31 run -> /run
drwxr-xr-x 1 root root      4096 Aug  3 20:53 spool
drwxrwxrwt 2 root root      4096 Jul 27 15:31 tmp
drwxr-xr-x 3 root root      4096 Aug  3 20:53 www

And that indeed as set by the compose the user abc has uid=1004:

9ad4a2eb6d1f:/var$ whoami
abc
9ad4a2eb6d1f:/var$ id -u abc
1004

So I decided to test if I could write to it. Sure I can...

9ad4a2eb6d1f:/var$ cd puter
9ad4a2eb6d1f:/var/puter$ ls -la
total 12
drwxrwxr-x 2 abc  kasm-user 4096 Aug 10 17:35 .
drwxr-xr-x 1 root root      4096 Aug 10 17:52 ..
9ad4a2eb6d1f:/var/puter$ touch test
9ad4a2eb6d1f:/var/puter$ ls -la
total 12
drwxrwxr-x 2 abc  kasm-user 4096 Aug 10 17:55 .
drwxr-xr-x 1 root root      4096 Aug 10 17:52 ..
-rw-r--r-- 1 abc  abc          0 Aug 10 17:55 test

So I can, which leaves me even more confused now. If I have a different container with the same uid that can write to it, why would puter not be able to do that?
Could it be a username issue? I have the host OS setting container as the owner of the directory. But have the container user abc, with the same uid=1004 try to access it. Would that be the issue? Thanks for the help!

@mikesellt
Copy link

Has there been an update to this issue? I'm having the same problem. I'm running this on a Synology NAS using Container Manager (which is basically a Docker Compose frontend). I have tried various changes to the compose file, but I haven't gotten this to work either. Getting the same error:

-- | -- | --
2024/09/10 01:36:45 | stdout |  
2024/09/10 01:36:45 | stdout | �[31;1m┗━━ [ END HELP ]�[0m
2024/09/10 01:36:45 | stdout | �[31;1m┃�[0m Help can be added in src/errors/error_help_details.
2024/09/10 01:36:45 | stdout | �[31;1m┃�[0m No help is available for this error.
2024/09/10 01:36:45 | stdout | �[31;1m┃�[0m Whoops! Looks like something isn't working!
2024/09/10 01:36:45 | stdout | �[31;1m┏━━ [ HELP:�[0m Cannot write to path: /var/puter �[31;1m]�[0m
2024/09/10 01:36:45 | stdout |  
2024/09/10 01:36:45 | stdout | at async file:///opt/puter/app/tools/run-selfhosted.js:157:9
2024/09/10 01:36:45 | stdout | at main (file:///opt/puter/app/tools/run-selfhosted.js:96:7)
2024/09/10 01:36:45 | stdout | at Kernel.boot (/opt/puter/app/src/backend/src/Kernel.js:70:14)
2024/09/10 01:36:45 | stdout | at Kernel.runtime_init (/opt/puter/app/src/backend/src/Kernel.js:62:40)
2024/09/10 01:36:45 | stdout | at RuntimeEnvironment.init (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:210:25)
2024/09/10 01:36:45 | stdout | at RuntimeEnvironment.init
 (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:235:37)
2024/09/10 01:36:45 | stdout | at RuntimeEnvironment.get_first_suitable_path_ (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:338:32)
2024/09/10 01:36:45 | stdout | at require_write_permission (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:78:19)
2024/09/10 01:36:45 | stdout | �[31;1m[BOOT/ERROR]�[0m Error: Cannot write to path: /var/puter
2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing require_write_permission on path /var/puter...
2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing  on path `/var/puter`... 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing `skip_if_not_exists` on path `/var/puter`... 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m Checking path `/var/puter` for working directory... 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m -> `require_if_not_undefined` doesn't like this path 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing `require_if_not_undefined` on path [undefined]... 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m Checking path `$RUNTIME_PATH` for working directory... 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m �[33;1mUSING�[0m `/etc/puter` for configuration. 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing `require_read_permission` on path `/etc/puter`... 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing `require_if_not_undefined` on path `/etc/puter`... 2024/09/10 01:36:45 | stdout | �[36;1m[BOOT/INFO]�[0m Checking path `$CONFIG_PATH` for configuration... 2024/09/10 01:36:45 | stdout | �[36;1mBoot logger started :)�[0m 2024/09/10 01:36:45 | stdout |   2024/09/10 01:36:45 | stdout | > node ./tools/run-selfhosted.js 2024/09/10 01:36:45 | stdout | > [email protected] start 2024/09/10 01:36:45 | stdout |   2024/09/10 01:36:16 | stdout |   2024/09/10 01:36:16 | stdout | �[31;1m┗━━ [ END HELP ]�[0m 2024/09/10 01:36:16 | stdout | �[31;1m┃�[0m Help can be added in src/errors/error_help_details. 2024/09/10 01:36:16 | stdout | �[31;1m┃�[0m No help is available for this error. 2024/09/10 01:36:16 | stdout | �[31;1m┃�[0m Whoops! Looks like something isn't working! 2024/09/10 01:36:16 | stdout | �[31;1m┏━━ [ HELP:�[0m `Cannot write to path: /var/puter` �[31;1m]�[0m 2024/09/10 01:36:16 | stdout |   2024/09/10 01:36:16 | stdout | at async file:///opt/puter/app/tools/run-selfhosted.js:157:9 2024/09/10 01:36:16 | stdout | at main (file:///opt/puter/app/tools/run-selfhosted.js:96:7) 2024/09/10 01:36:16 | stdout | at Kernel.boot (/opt/puter/app/src/backend/src/Kernel.js:70:14) 2024/09/10 01:36:16 | stdout | at Kernel._runtime_init (/opt/puter/app/src/backend/src/Kernel.js:62:40) 2024/09/10 01:36:16 | stdout | at RuntimeEnvironment.init (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:210:25) 2024/09/10 01:36:16 | stdout | at RuntimeEnvironment.init_ (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:235:37) 2024/09/10 01:36:16 | stdout | at RuntimeEnvironment.get_first_suitable_path_ (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:338:32) 2024/09/10 01:36:16 | stdout | at require_write_permission (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:78:19) 2024/09/10 01:36:16 | stdout | �[31;1m[BOOT/ERROR]�[0m Error: Cannot write to path: /var/puter 2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing `require_write_permission` on path `/var/puter`... 2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing  on path /var/puter...
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing skip_if_not_exists on path /var/puter...
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m Checking path /var/puter for working directory...
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m -> require_if_not_undefined doesn't like this path
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing require_if_not_undefined on path [undefined]...
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m Checking path $RUNTIME_PATH for working directory...
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m �[33;1mUSING�[0m /etc/puter for configuration.
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing require_read_permission on path /etc/puter...
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m -> doing require_if_not_undefined on path /etc/puter...
2024/09/10 01:36:16 | stdout | �[36;1m[BOOT/INFO]�[0m Checking path $CONFIG_PATH for configuration...
2024/09/10 01:36:16 | stdout | �[36;1mBoot logger started :)�[0m
2024/09/10 01:36:16 | stdout |  
2024/09/10 01:36:16 | stdout | > node ./tools/run-selfhosted.js
2024/09/10 01:36:16 | stdout | > [email protected] start

@capira
Copy link

capira commented Sep 21, 2024

Hello, I have the same error. I don't know if there is any solution.

I have created a system user without home named puter:

$ id puter
uid=993(puter) gid=990(puter) grupos=990(puter),994(docker)

I have created the docker compose file with two services: puter and webtop as @francescoferri indicated.

$ cat docker-compose.yml
version: "3.8"
services:
  puter:
    container_name: puter
    image: ghcr.io/heyputer/puter:latest
    pull_policy: always
    restart: unless-stopped
    ports:
      - '4100:4100'
    environment:
      - PUID=993
      - PGID=990
      - TZ=Europe/Madrid
    volumes:
      - /var/puter/config:/etc/puter
      - /var/puter/data:/var/puter
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 30s

  webtop:
    image: lscr.io/linuxserver/webtop:latest
    container_name: webtop
    security_opt:
      - seccomp:unconfined #optional
    environment:
      - PUID=993
      - PGID=990
      - TZ=Europe/Madrid
      - TITLE=Webtop #optional
    volumes:
      - /var/puter/webtop/data:/config
      - /var/puter/config:/etc/puter
      - /var/puter/data:/var/puter
    ports:
      - 3000:3000
      - 3001:3001
    shm_size: "1gb" #optional
    restart: unless-stopped

Puter gives error accessing /var/puter, however if I connect to webtop I can access and write without problems.

$ docker exec -it puter sh
Error response from daemon: Container d9ef518aa6599f7a4d297e09e271e669c044af44211dab29f3306e9bcd1d4c4b is restarting, wait until the container is running

$ sudo -u puter docker logs puter
> [email protected] start
> node ./tools/run-selfhosted.js

Boot logger started :)
[BOOT/INFO] Checking path `$CONFIG_PATH` for configuration...
[BOOT/INFO] -> doing `require_if_not_undefined` on path [undefined]...
[BOOT/INFO] -> `require_if_not_undefined` doesn't like this path
[BOOT/INFO] Checking path `/etc/puter` for configuration...
[BOOT/INFO] -> doing `skip_if_not_exists` on path `/etc/puter`...
[BOOT/INFO] -> doing `require_read_permission` on path `/etc/puter`...
[BOOT/INFO] USING `/etc/puter` for configuration.
[BOOT/INFO] Checking path `$RUNTIME_PATH` for working directory...
[BOOT/INFO] -> doing `require_if_not_undefined` on path [undefined]...
[BOOT/INFO] -> `require_if_not_undefined` doesn't like this path
[BOOT/INFO] Checking path `/var/puter` for working directory...
[BOOT/INFO] -> doing `skip_if_not_exists` on path `/var/puter`...
[BOOT/INFO] -> doing `` on path `/var/puter`...
[BOOT/INFO] -> doing `require_write_permission` on path `/var/puter`...
[BOOT/ERROR] Error: Cannot write to path: /var/puter
    at require_write_permission (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:78:19)
    at RuntimeEnvironment.get_first_suitable_path_ (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:338:32)
    at RuntimeEnvironment.init_ (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:235:37)
    at RuntimeEnvironment.init (/opt/puter/app/src/backend/src/boot/RuntimeEnvironment.js:210:25)
    at Kernel._runtime_init (/opt/puter/app/src/backend/src/Kernel.js:62:40)
    at Kernel.boot (/opt/puter/app/src/backend/src/Kernel.js:70:14)
    at main (file:///opt/puter/app/tools/run-selfhosted.js:96:7)
    at async file:///opt/puter/app/tools/run-selfhosted.js:157:9

┏━━ [ HELP: `Cannot write to path: /var/puter` ]
┃ Whoops! Looks like something isn't working!
┃ No help is available for this error.
┃ Help can be added in src/errors/error_help_details.
┗━━ [ END HELP ]
$ docker exec -it webtop sh
root@c8ba85a917b6:/# ls -la /var
total 72
drwxr-xr-x 1 root root 4096 Sep 21 19:57 .
drwxr-xr-x 1 root root 4096 Sep 21 19:57 ..
drwxr-xr-x 1 root root 4096 Sep 14 20:48 cache
drwxr-xr-x 2 root root 4096 Sep 14 20:48 db
dr-xr-xr-x 2 root root 4096 Sep 14 15:32 empty
drwxr-xr-x 1 root root 4096 Sep 17 05:54 lib
drwxr-xr-x 2 root root 4096 Sep 14 15:32 local
drwxr-xr-x 3 root root 4096 Sep 14 15:32 lock
drwxr-xr-x 1 root root 4096 Sep 14 20:48 log
drwxr-xr-x 2 root root 4096 Sep 14 15:32 mail
drwxr-xr-x 2 root root 4096 Sep 14 15:32 opt
drwxrwxr-x 2 abc  abc  4096 Sep 21 19:49 puter
lrwxrwxrwx 1 root root    4 Sep 14 15:32 run -> /run
drwxr-xr-x 1 root root 4096 Sep 14 20:48 spool
drwxrwxrwt 2 root root 4096 Sep 14 15:32 tmp
drwxr-xr-x 3 root root 4096 Sep 14 20:48 www

root@c8ba85a917b6:/# sudo -u abc id
uid=993(abc) gid=990(abc) groups=990(abc),10(wheel),103(docker),1000(users)

root@c8ba85a917b6:/# sudo -u abc touch /var/puter/b
root@c8ba85a917b6:/# sudo -u abc ls -la /var/puter
total 12
drwxrwxr-x 2 abc  abc  4096 Sep 21 19:59 .
drwxr-xr-x 1 root root 4096 Sep 21 19:57 ..
-rw-r--r-- 1 abc  abc     0 Sep 21 19:49 a
-rw-r--r-- 1 abc  abc     0 Sep 21 19:59 b

Thanks in advance!

@capira
Copy link

capira commented Sep 21, 2024

Reviewing the issue #209 I have tried assigning user 1000 and group 1000 permissions and configuring docker compose with these and it works.

Is it possible that these permissions are written in the code? is it not possible to parameterize them, @KernelDeimos ?

In any case, thank you very much for the awesome work.

@DieterKoblenz
Copy link

DieterKoblenz commented Nov 5, 2024

Unfortunately, I have the same problem, since I can't create a UID 1000 / GID 1000 user I am stuck. Shame that this is not possible yet. It appears indeed that docker parameters are not used.

Only when I give "everyone" access to the folder it will work.

@KernelDeimos
Copy link
Contributor

Reviewing the issue #209 I have tried assigning user 1000 and group 1000 permissions and configuring docker compose with these and it works.

Is it possible that these permissions are written in the code? is it not possible to parameterize them, @KernelDeimos ?

In any case, thank you very much for the awesome work.

Parameterizing the UID/GID of the user in the Docker container would be tricky. I think we need to mount this in a different way. Docker must have a way to make this easier, but I don't know what it is yet.

Neither of the maintainers (myself included) use Docker all that much. I know my way around a Dockerfile and the cli, but I don't my way around all the features or what people usually expect when deploying a docker image, so a lot of these issues that come up confuse me. For example, I don't understand why setting ownership to 1000:1000 causes a problem for some people, and it would take me much longer to improve that experience than someone who does this sort of thing all the time.

@KernelDeimos KernelDeimos added the help wanted Extra attention is needed label Nov 7, 2024
@KernelDeimos KernelDeimos changed the title [BOOT/ERROR] Error: Cannot write to path: /var/puter [docker] A way mount the data directory that doesn't require chown Nov 7, 2024
@KernelDeimos
Copy link
Contributor

I added the help wanted label, and changed the title to something a Docker-guru who knows how to solve this will recognize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

6 participants