Web SSH: File up/download buttons greyed out #44864

uedvt359 · 2024-07-31T11:12:17Z

uedvt359
Jul 31, 2024

I don't know what I'm doing wrong, but these buttons are greyed out on all my hosts. My roles all have ssh_file_copy enabled, and tsh scp works.
I enrolled them into teleport manually (creating a resource.yaml and role.yaml), and installed the openssh certificate authority in the sshd_config, so there is no agent running on my servers.

Answered by uedvt359

Aug 8, 2024

I managed to figure out what I was doing wrong, and it was quite a doozy: my node resource has a metadata.name field, in the form of hostname:port. i am generating them from a script/template, and pipe them straight into tctl create -f (which uses the metadata.name to match up which host i am updating).

this field has a very surprising behaviour:

if it is exactly the hostname (no port), web ssh and web-upload/download work
if it is a uuid, both work as well
if it is hostname:port, web ssh itself works, but upload/download is broken (this might be related to the "unable to resolve hostname" error i posted above)
if it is anything else, nothing works ("direct dialling is not supported")

View full answer

webvictim · 2024-07-31T15:34:34Z

webvictim
Jul 31, 2024
Collaborator

Can you share the full YAML output of every role that your user has? You can find the full list of roles in the "Users" section of the web UI:

AFAIK there's nothing that would block you from using file upload/download other than an RBAC permission being denied somewhere.

5 replies

uedvt359 Aug 1, 2024
Author

sure. my user has the default editor and auditor roles, as well as my custom 'access-all-ssh-nodes'. (i used tctl get roles/$rolename to export them)

kind: role
metadata:
  id: 7876710618521667430
  name: my-ssh-access
  revision: 6607882f-93b2-4fed-94fc-4b8a5fee02fc
spec:
  allow:
    logins:
    - ansible
    - root
    node_labels:
      sshname:
      - FQDN:22 for a whole bunch of nodes
  deny: {}
  options:
    cert_format: standard
    create_db_user: false
    create_desktop_user: false
    desktop_clipboard: true
    desktop_directory_sharing: true
    enhanced_recording:
    - command
    - network
    forward_agent: false
    idp:
      saml:
        enabled: true
    max_session_ttl: 30h0m0s
    permit_x11_forwarding: true
    pin_source_ip: false
    port_forwarding: true
    record_session:
      default: best_effort
      desktop: true
    require_session_mfa: true
    ssh_file_copy: true
version: v7

auditor

kind: role
metadata:
  description: Review cluster events and replay sessions
  id: 7945949270858023143
  labels:
    teleport.internal/resource-type: preset
  name: auditor
  revision: e740c42c-c3ae-456e-a263-83db2c816120
spec:
  allow:
    logins:
    - no-login-59b7be70-4421-4e85-8478-4c43c5f48363
    rules:
    - resources:
      - session
      verbs:
      - list
      - read
    - resources:
      - event
      verbs:
      - list
      - read
    - resources:
      - session_tracker
      verbs:
      - list
      - read
    - resources:
      - cluster_alert
      verbs:
      - list
      - read
    - resources:
      - instance
      verbs:
      - list
      - read
    - resources:
      - security_report
      verbs:
      - list
      - read
      - use
    - resources:
      - audit_query
      verbs:
      - list
      - read
      - use
  deny: {}
  options:
    cert_format: standard
    create_db_user: false
    create_desktop_user: false
    desktop_clipboard: true
    desktop_directory_sharing: true
    enhanced_recording:
    - command
    - network
    forward_agent: false
    idp:
      saml:
        enabled: true
    max_session_ttl: 30h0m0s
    pin_source_ip: false
    port_forwarding: true
    record_session:
      desktop: false
    ssh_file_copy: true
version: v7

editor

kind: role
metadata:
  description: Edit cluster configuration
  id: 1966442169018747912
  labels:
    teleport.internal/resource-type: preset
  name: editor
  revision: 080cb6eb-bb34-4a1b-acca-4ed271dbd748
spec:
  allow:
    rules:
    - resources:
      - user
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - role
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - bot
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - oidc
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - saml
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - github
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - oidc_request
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - saml_request
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - github_request
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_audit_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_auth_preference
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - auth_connector
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_name
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_networking_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - session_recording_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - external_audit_storage
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - ui_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - trusted_cluster
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - remote_cluster
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - token
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - connection_diagnostic
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - db
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - database_certificate
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - installer
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - device
      verbs:
      - list
      - create
      - read
      - update
      - delete
      - create_enroll_token
      - enroll
    - resources:
      - db_service
      verbs:
      - list
      - read
    - resources:
      - instance
      verbs:
      - list
      - read
    - resources:
      - login_rule
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - saml_idp_service_provider
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - user_group
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - plugin
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - okta_import_rule
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - okta_assignment
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - assistant
      verbs:
      - list
      - create
      - read
      - update
      - delete
      - use
    - resources:
      - lock
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - integration
      verbs:
      - list
      - create
      - read
      - update
      - delete
      - use
    - resources:
      - billing
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - cluster_alert
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - access_list
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - node
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - discovery_config
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - security_report
      verbs:
      - list
      - create
      - read
      - update
      - delete
      - use
    - resources:
      - audit_query
      verbs:
      - list
      - create
      - read
      - update
      - delete
      - use
    - resources:
      - access_graph
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - server_info
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - db_object_import_rule
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - app_server
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - access_monitoring_rule
      verbs:
      - list
      - create
      - read
      - update
      - delete
    - resources:
      - crown_jewel
      verbs:
      - list
      - create
      - read
      - update
      - delete
  deny: {}
  options:
    cert_format: standard
    create_db_user: false
    create_desktop_user: false
    desktop_clipboard: true
    desktop_directory_sharing: true
    enhanced_recording:
    - command
    - network
    forward_agent: true
    idp:
      saml:
        enabled: true
    max_session_ttl: 30h0m0s
    pin_source_ip: false
    port_forwarding: true
    record_session:
      desktop: false
    ssh_file_copy: true
version: v7

uedvt359 Aug 6, 2024
Author

hey, @webvictim, I'm wondering what else I can do to debug this? I threw away my teleport.yaml and database and tried again with a minimal config, still doesn't work.

or is this feature incompatible with sub_kind: openssh nodes? (that's the only kind I got)

webvictim Aug 6, 2024
Collaborator

It works fine for me on an OpenSSH node with the following config:

kind: node
metadata:
  labels:
    type: openssh
  name: c70aeeed-84d3-4bd2-a739-88e70f0f344e
  revision: 00000000-0000-0000-0000-000000000000
spec:
  addr: 172.31.8.63:22
  hostname: ip-172-31-8-63
  rotation:
    current_id: ""
    last_rotated: "2021-12-03T19:43:59.014196449Z"
    schedule:
      standby: "0001-01-01T00:00:00Z"
      update_clients: "0001-01-01T00:00:00Z"
      update_servers: "0001-01-01T00:00:00Z"
    started: "0001-01-01T00:00:00Z"
  version: ""
sub_kind: openssh
version: v2

uedvt359 Aug 6, 2024
Author

that's exactly the same node config as I have (except for some different labels, and metadata.id is automatically populated). would you mind sharing a /etc/teleport.yaml please? i have attached mine

/etc/teleport.yaml

version: v3
teleport:
  nodename: myserver.com
  data_dir: /var/lib/teleport
  log:
    output: stderr
    severity: INFO
    format:
      output: text
  diag_addr: 0.0.0.0:3000
  storage:
    type: postgresql
    conn_string: postgresql://[email protected]:5432/teleport_backend?pool_max_conns=20
    audit_events_uri: postgresql://teleport_dbuser@/teleport_audit
    audit_sessions_uri: file:///var/lib/teleport/log
auth_service:
  enabled: "yes"
  listen_addr: 0.0.0.0:3025
  cluster_name: myteleport.org
  proxy_listener_mode: multiplex
  session_recording: "off"
  proxy_checks_host_keys: no
  authentication:
    type: local
    second_factor: webauthn
    webauthn:
      rp_id: myteleport.org
      attestation_allowed_cas:
      - "/var/lib/teleport/yubi_attestation.pem"
    connector_name: passwordless
ssh_service:
  enabled: "yes"
proxy_service:
  enabled: "yes"
  web_listen_addr: 0.0.0.0:443
  public_addr: myteleport.org:443
  https_keypairs: []
  https_keypairs_reload_interval: 0s
  acme:
    enabled: "yes"
app_service:
    enabled: yes
    resources:
    - labels:
        "*": "*"
windows_desktop_service:
  enabled: "no"
db_service:
  enabled: "yes"
  resources:
  - labels:
      "*": "*"
kubernetes_service:
  enabled: "no"

webvictim Aug 6, 2024
Collaborator

Mine is pretty standard and boring, there isn't much to it:

teleport:
  log:
    output: stderr
    severity: INFO
    format:
      output: text
  data_dir: /var/lib/teleport
  diag_addr: 0.0.0.0:3434

auth_service:
  enabled: yes
  authentication:
    type: github
    # require webauthn only (also enables MFA for tctl actions)
    second_factor: webauthn
    webauthn:
      rp_id: teleport.example.com
  cluster_name: purple

proxy_service:
  enabled: yes
  web_listen_addr: 0.0.0.0:443
  public_addr: teleport.example.com:443
  acme:
    enabled: true
    email: "[email protected]"

ssh_service:
  enabled: yes
  commands:
  - name: hostname
    command: [hostname]
    period: 1h0s
  - name: version
    command: ['/usr/local/bin/teleport', 'version']
    period: 30m0s
  - name: external_ip
    command: ['curl', 'icanhazip.com']
    period: 1h0ms

uedvt359 · 2024-08-01T11:14:40Z

uedvt359
Aug 1, 2024
Author

I have disabled session MFA (in case it was the culprit), did not help. I did notice the following error in the logs, though, directly after connecting:

2024-08-01T13:07:50+02:00 INFO [WEB]       "Generating new session for myteleport.org\n" web/apiserver.go:3296
2024-08-01T13:07:50+02:00 INFO [WEB]       Getting terminal to web.TerminalRequest{Server:"myserver.org:22", Login:"root", Term:session.TerminalParams{W:236, H:46}, SessionID:"", ProxyHostPort:"", InteractiveCommand:[]string(nil), KeepAliveInterval:0, ParticipantMode:"peer"}. web/apiserver.go:3288
2024-08-01T13:07:50+02:00 INFO [AUDIT]     cert.create cert_type:user cluster_name:myteleport.org code:TC000I ei:0 event:cert.create disallow_reissue:true expires:2024-08-01T23:07:29.004181365Z logins:[root -teleport-internal-join] prev_identity_expires:0001-01-01T00:00:00Z private_key_policy:none roles:[ssh-access editor] route_to_cluster:myteleport.org teleport_cluster:myteleport.org aws_role_arns:<nil> azure_identities:<nil> db_names:<nil> db_roles:<nil> db_users:[] gcp_service_accounts:<nil> host_user_gid:[] host_user_uid:[] kubernetes_groups:<nil> kubernetes_users:<nil> logins:[] windows_logins:[myusername] user:myusername time:2024-08-01T11:07:50.274Z uid:927584a0-e8b3-4cd8-b0d6-dfab66e7ffb6 user_agent:grpc-go/1.63.2 events/emitter.go:288
2024-08-01T13:07:50+02:00 WARN [NODE:FORW] Insecure configuration! Strict host key checking disabled, allowing login without checking host key of type ecdsa-sha2-nistp256. srv/authhandlers.go:572
2024-08-01T13:07:50+02:00 INFO [NODE:FORW] Creating (interactive) session 9e85d711-e962-4447-9b3f-de7467c39535. id:3 local:1.2.3.4:22 login:root remote:192.168.1.10:60620 teleportUser:myusername srv/sess.go:341
2024-08-01T13:07:50+02:00 INFO [AUDIT]     session.start addr.local:1.2.3.4:22 addr.remote:192.168.1.10:60620 cluster_name:myteleport.org code:T2000I ei:0 event:session.start forwarded_by:2856c7bd-d933-4473-a3a2-9ce2f0fb7975 initial_command:[] login:root namespace:default private_key_policy:none proto:ssh server_addr:1.2.3.4:22 server_hostname:myserver.org server_id:myserver.org:22.myteleport.org server_sub_kind:openssh server_version:15.4.11 session_recording:proxy sid:9e85d711-e962-4447-9b3f-de7467c39535 size:80:25 time:2024-08-01T11:07:50.376Z uid:98663e62-0b5b-4f70-a77f-90d768bcc6c3 user:myusername user_kind:1 events/emitter.go:288
2024-08-01T13:07:50+02:00 INFO [SESSION:N] New party ServerContext(192.168.1.10:60620->1.2.3.4:22, user=root, id=3) party(id=012b10e8-8836-488c-ae84-44ac802d9ddb) joined the session with participant mode: peer. session_id:9e85d711-e962-4447-9b3f-de7467c39535 srv/sess.go:1928

2024-08-01T13:07:50+02:00 WARN [WEBSOCKET] Failure sending session data error:[
ERROR REPORT:
Original Error: *trace.NotFoundError unable to resolve hostname for server myserver.org
Stack Trace:
        github.com/gravitational/teleport/lib/web/apiserver.go:3273 github.com/gravitational/teleport/lib/web.(*Handler).siteNodeConnect.func1
        github.com/gravitational/teleport/lib/web/terminal.go:379 github.com/gravitational/teleport/lib/web.(*TerminalHandler).writeSessionData
        github.com/gravitational/teleport/lib/web/terminal.go:837 github.com/gravitational/teleport/lib/web.(*TerminalHandler).streamTerminal.func3
        runtime/asm_amd64.s:1695 runtime.goexit
User Message: unable to resolve hostname for server myserver.org] session_id:b69aae4f-615f-481c-98a6-3966153717e9 web/terminal.go:838

2024-08-01T13:07:50+02:00 INFO [AUDIT]     session.start addr.local:1.2.3.4:22 addr.remote:192.168.1.10:60620 cluster_name:myteleport.org code:T2000I ei:0 event:session.start forwarded_by:2856c7bd-d933-4473-a3a2-9ce2f0fb7975 initial_command:[] login:root namespace:default private_key_policy:none proto:ssh server_addr:1.2.3.4:22 server_hostname:myserver.org server_id:myserver.org:22.myteleport.org server_sub_kind:openssh server_version:15.4.11 session_recording:proxy sid:9e85d711-e962-4447-9b3f-de7467c39535 size:80:25 time:2024-08-01T11:07:50.376Z uid:98663e62-0b5b-4f70-a77f-90d768bcc6c3 user:myusername user_kind:1 events/emitter.go:288

calling host myserver.org on the myteleport.org host correctly returns the ip address, so i'm not sure why it says it can't resolve it.

2 replies

webvictim Aug 6, 2024
Collaborator

Can you share the full Teleport config for your auth/proxy? You can redact the hostnames but don't take too much out, it might be relevant.

Also, how is your Teleport cluster deployed? What hosts it? Does it have any load balancers or reverse proxies in front of it?

uedvt359 Aug 6, 2024
Author

just did, see comment above :)

uedvt359 · 2024-08-08T14:49:33Z

uedvt359
Aug 8, 2024
Author

I managed to figure out what I was doing wrong, and it was quite a doozy: my node resource has a metadata.name field, in the form of hostname:port. i am generating them from a script/template, and pipe them straight into tctl create -f (which uses the metadata.name to match up which host i am updating).

this field has a very surprising behaviour:

if it is exactly the hostname (no port), web ssh and web-upload/download work
if it is a uuid, both work as well
if it is hostname:port, web ssh itself works, but upload/download is broken (this might be related to the "unable to resolve hostname" error i posted above)
if it is anything else, nothing works ("direct dialling is not supported")

2 replies

uedvt359 Aug 8, 2024
Author

thanks again, @/webvictim, for your time.

webvictim Aug 8, 2024
Collaborator

No problem, I'm glad you figured it out and posted an explanation!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Web SSH: File up/download buttons greyed out #44864

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 9 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Web SSH: File up/download buttons greyed out #44864

uedvt359 Jul 31, 2024

Replies: 3 comments · 9 replies

webvictim Jul 31, 2024 Collaborator

uedvt359 Aug 1, 2024 Author

uedvt359 Aug 6, 2024 Author

webvictim Aug 6, 2024 Collaborator

uedvt359 Aug 6, 2024 Author

webvictim Aug 6, 2024 Collaborator

uedvt359 Aug 1, 2024 Author

webvictim Aug 6, 2024 Collaborator

uedvt359 Aug 6, 2024 Author

uedvt359 Aug 8, 2024 Author

uedvt359 Aug 8, 2024 Author

webvictim Aug 8, 2024 Collaborator

uedvt359
Jul 31, 2024

Replies: 3 comments 9 replies

webvictim
Jul 31, 2024
Collaborator

uedvt359 Aug 1, 2024
Author

uedvt359 Aug 6, 2024
Author

webvictim Aug 6, 2024
Collaborator

uedvt359 Aug 6, 2024
Author

webvictim Aug 6, 2024
Collaborator

uedvt359
Aug 1, 2024
Author

webvictim Aug 6, 2024
Collaborator

uedvt359 Aug 6, 2024
Author

uedvt359
Aug 8, 2024
Author

uedvt359 Aug 8, 2024
Author

webvictim Aug 8, 2024
Collaborator