access denied to user connecting to X-X-X-X-X

[Smartich0ke]

When I try to connect to a server through the Teleport web UI by clicking connect and entering an OS user in the text box, I get the following error:

access denied to user connecting to X-X-X-X-X

However, when I select the user through the drop-down menu that appears under the text box, the connection succeeds. But for some reason, the list of users is incomplete, and often doesn't have the user I need to connect to the server in the menu, or has users for another server. This leaves me unable to connect to most of my servers.

tsh/tctl version string:

Teleport v11.2.1 git:api/v11.2.1-0-g999ff61 go1.19.5
Proxy version: 11.2.1

I'm running Teleport in Docker and the servers I'm connecting to are various OSes, mostly Debian and Ubuntu.

[webvictim]

The error is appearing because your Teleport RBAC permissions do not currently grant you access to the servers you're trying to connect to as the login you're trying to use.

Can you share the content of all roles that are assigned to your Teleport user?

The output of tctl get user/<teleport_username> from your Teleport auth server would also be helpful.

[Smartich0ke]

I have the access role assigned to myself:

kind: role
metadata:
  description: Access cluster resources
  id: 1674161930691441615
  name: access
spec:
  allow:
    app_labels:
      '*': '*'
    aws_role_arns:
    - '{{internal.aws_role_arns}}'
    db_labels:
      '*': '*'
    db_names:
    - '{{internal.db_names}}'
    db_users:
    - '{{internal.db_users}}'
    kubernetes_groups:
    - '{{internal.kubernetes_groups}}'
    kubernetes_labels:
      '*': '*'
    kubernetes_users:
    - '{{internal.kubernetes_users}}'
    logins:
    - '{{internal.logins}}'
    node_labels:
      '*': '*'
    rules:
    - resources:
      - event
      verbs:
      - list
      - read
    - resources:
      - session
      verbs:
      - read
      - list
      where: contains(session.participants, user.metadata.name)
    - resources:
      - instance
      verbs:
      - list
      - read
    windows_desktop_labels:
      '*': '*'
    windows_desktop_logins:
    - '{{internal.windows_logins}}'
  deny: {}
  options:
    cert_format: standard
    create_host_user: false
    desktop_clipboard: true
    desktop_directory_sharing: true
    enhanced_recording:
    - command
    - network
    forward_agent: true
    max_session_ttl: 30h0m0s
    pin_source_ip: false
    port_forwarding: true
    record_session:
      desktop: true
    ssh_file_copy: true
version: v5

and the editor role:

kind: role
metadata:
  description: Edit cluster configuration
  id: 1674161930603488036
  name: editor
spec:
  allow:
    rules:
    - resources:
      - user
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - role
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - oidc
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - saml
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - github
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - oidc_request
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - saml_request
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - github_request
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_audit_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_auth_preference
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - auth_connector
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_name
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_networking_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - session_recording_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - trusted_cluster
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - remote_cluster
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - token
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - connection_diagnostic
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - db
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - database_certificate
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - installer
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - instance
      verbs:
      - list
      - read
  deny: {}
  options:
    cert_format: standard
    create_host_user: false
    desktop_clipboard: true
    desktop_directory_sharing: true
    enhanced_recording:
    - command
    - network
    forward_agent: true
    max_session_ttl: 30h0m0s
    pin_source_ip: false
    port_forwarding: true
    record_session:
      desktop: false
    ssh_file_copy: true
version: v5

The output of tctl get user/myuser:

kind: user
metadata:
  id: 1674094049269338790
  name: npatrick1
spec:
  created_by:
    time: "2023-01-18T20:15:21.81720796Z"
    user:
      name: 8469815c-8053-417b-bf6f-e59075564dc4.localhost
  expires: "0001-01-01T00:00:00Z"
  roles:
  - access
  - editor
  status:
    is_locked: false
    lock_expires: "0001-01-01T00:00:00Z"
    locked_time: "0001-01-01T00:00:00Z"
    recovery_attempt_lock_expires: "0001-01-01T00:00:00Z"
  traits:
    aws_role_arns: null
    db_names: null
    db_users: null
    kubernetes_groups: null
    kubernetes_users: null
    logins:
    - root
    - npatrick
    - debian-vhost-1
    - debian-vhost-2
    - debian-vhost-3
    - ""
    windows_logins: null
version: v2

[webvictim]

With this setup, you should be able to SSH to any server as any of these users: root, npatrick, debian-vhost-1, debian-vhost-2, debian-vhost-3

If you try to connect as any user which is not in that list, you will see the access denied error you originally reported.

[Smartich0ke]

ah ok thanks for the info. But now I'm just getting a message saying disconnected ?

[webvictim]

Do you see anything in the Teleport logs (either on the Teleport auth/proxy or on the nodes themselves) when that message appears?

[Smartich0ke]

Oh sorry I just forgot to enable websockets in my reverse proxy conf. It seems to be working now. Thanks for your help!

[iercan]

I was getting same error and it turns out it is because of server's time wasn't sync with teleport proxy. I installed ntp and synced time and problem solved. Hope this helps to anybody else.

[shearern]

That was it for me too. Had to set the timezone the same.

[anilrajrimal1]

I have tried setting up my TZ but it did't worked for me??
are there any other solutions related to this?

[webvictim]

It isn't about changing the timezone. You have to make sure that your server is running a time-sync service like ntp or chrony.

Assuming you're on a relatively modern system with systemd you can check the output of timedatectl:

gus@hades:~$ timedatectl
               Local time: Thu 2024-08-29 10:22:01 ADT
           Universal time: Thu 2024-08-29 13:22:01 UTC
                 RTC time: Thu 2024-08-29 13:22:01
                Time zone: America/Halifax (ADT, -0300)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

It should show "System clock synchronized: yes".

If it doesn't, consult a guide online about how to enable time sync for your operating system.

[anilrajrimal1]

can you please explain me more about this?
why it is required?
how actually this is related to teleport?

[webvictim]

It's required because Teleport uses and issues short lived certificates for all its operations. Certificates have "valid from" (not before) and "valid until" (not after) dates built into them.

If the time synchronisation is not accurate between all clients and servers, the certificates will appear to be not yet valid or expired on some servers and not on others, even though the real time is actually the same.