[BUG] [9.7_p1-r4-ls174] is broken, sshd not started

[xuan-w]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

sshd not started in the latest image [9.7_p1-r4-ls174] but old image [9.7_p1-r4-ls173] works fine.

Expected Behavior

sshd started.

Steps To Reproduce

1. pull the latest image
2. start the docker with docker-compose up
3. terminal shows "sshd is listening on port 2222"
4. but sshd is not running (can be verified by docker exec -it)

Environment

- OS:Debian 12
- How docker service was installed: 
deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian   bookworm stable

CPU architecture

x86-64

Docker creation

version: "2.1"
services:
  openssh-server:
    #image: lscr.io/linuxserver/openssh-server:9.7_p1-r4-ls173
    image: lscr.io/linuxserver/openssh-server
    container_name: openssh-server
    hostname: openssh-server #optional
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - PUBLIC_KEY_DIR=/config/pub_keys
      - SUDO_ACCESS=false #optional
      - PASSWORD_ACCESS=false #optional
      - USER_NAME=xxxx
    volumes:
      - /somefolder/sshd:/config
    ports:
      - XXXX:2222
    restart: unless-stopped

Container logs

[migrations] started
[migrations] no migrations found
───────────────────────────────────────

      ██╗     ███████╗██╗ ██████╗
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝

   Brought to you by linuxserver.io
───────────────────────────────────────

To support LSIO projects visit:
https://www.linuxserver.io/donate/

───────────────────────────────────────
GID/UID
───────────────────────────────────────

User UID:    1000
User GID:    1000
───────────────────────────────────────
Linuxserver.io version: 9.7_p1-r4-ls174
Build-date: 2024-11-22T16:51:16+00:00
───────────────────────────────────────

User name is set to xxxx
sudo is disabled.
SSH host public key(s):
ssh-dss AAAAB3Nza...k0M= root@openssh-server
ecdsa-sha2-nistp256 AAAAE2...Q0KjA8= root@openssh-server
ssh-ed25519 AAAAC...maExsw6aAX root@openssh-server
ssh-rsa AAAAB3...WbBU= root@openssh-server
sshd is listening on port 2222
User/password ssh access is disabled.
[custom-init] No custom files found, skipping...

[github-actions]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[thespad]

Please provide the sshd logs from /config/logs/openssh/current

I can't replicate this issue with either a clean or existing install so it suggests something environmental at work.

[ionos]

Hi! I am experiencing a similar issue, at least similar symptoms. The issue for me seems to be related to the recent changes to the directory permissions of /config/ssh_host_keys. I am using a custom user (PUID/PGID/USER_NAME) and SUDO_ACCESS=false. The startup fails with /etc/ssh/sshd_config: Permission denied.

[thespad]

Please provide the sshd logs from /config/logs/openssh/current

I can't replicate this issue with either a clean or existing install so it suggests something environmental at work.

[ionos]

In my case, it's just 024-11-23 08:23:27.326626818 /etc/ssh/sshd_config: Permission denied over and over again.

I reverted back to 9.7_p1-r4-ls173, which works fine.

[xuan-w]

Please provide the sshd logs from /config/logs/openssh/current

I can't replicate this issue with either a clean or existing install so it suggests something environmental at work.

Thank you!

Here is the log

2024-11-24 04:21:47.008230623  Received signal 15; terminating.M
2024-11-24 04:22:02.912558749  /etc/ssh/sshd_config: Permission denied
2024-11-24 04:22:03.928418238  /etc/ssh/sshd_config: Permission denied
2024-11-24 04:22:04.942879424  /etc/ssh/sshd_config: Permission denied
2024-11-24 04:22:05.957094403  /etc/ssh/sshd_config: Permission denied
2024-11-24 04:22:06.972329416  /etc/ssh/sshd_config: Permission denied

[xuan-w]

Hi! I am experiencing a similar issue, at least similar symptoms. The issue for me seems to be related to the recent changes to the directory permissions of /config/ssh_host_keys. I am using a custom user (PUID/PGID/USER_NAME) and SUDO_ACCESS=false. The startup fails with /etc/ssh/sshd_config: Permission denied.

Yes, in ls173 that folder was owned by the user, in ls174 /config/ssh_host_keys is owned by root

drwxr-x--- 2 root  root  4096 Nov 24 04:36 ssh_host_keys

so this must be the reason.

[thespad]

Ugh, people have obviously been fiddling with their folder permissions, the default is drwxr-xr-x which would be working.

I'll do a PR to fix the permissions on the folder and keys for anyone that's changed them.

[AngellusMortis]

The PR fix for this issue caused other issues. The host keys are required to be 600.

2024-11-24 18:34:47.383875416  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2024-11-24 18:34:47.383879484  @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
2024-11-24 18:34:47.383881964  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2024-11-24 18:34:47.383884257  Permissions 0640 for '/etc/ssh/ssh_host_ed25519_key' are too open.
2024-11-24 18:34:47.383886589  It is required that your private key files are NOT accessible by others.
2024-11-24 18:34:47.383888524  This private key will be ignored.
2024-11-24 18:34:47.385395663  sshd: no hostkeys available -- exiting.
root@bastion-6f8cfb79f8-h5wj4:/# command terminated with exit code 137

[ionos]

The PR fix for this issue caused other issues. The host keys are required to be 600.

Is having a single user for the service itself and for the login like trying to square the circle?

In order to prevent a logged in user from doing something silly/harmful/unintended (incl. accessing the server's private key, changing the config, ...), it might be cleaner to run the service as a different user from the one being used for logins.

[thespad]

That's something you can't do unless sshd is running as root because it needs to setuid when a user other than the one running sshd tries to login. You either run as root or you're limited to logging in as the user that runs sshd.

#97 should fix the issue; fundamentally it's caused by sshd_config and the host keys sharing the same directory and needing different permissions, so splitting them up solves it without too much fuss.