Unable to login via tsh

[AritraWork97]

Hi,

When I try to login to the proxy server, using tsh login, it shows me an error of ERROR: object not found
The user is there I have checked.

Any help will be appreciated.

[christian-heusel]

I have the same problem with the following detailed info:

chris@pioneer ~ » tsh -d login --proxy=<host>
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/home/chris/.tsh/keys/<host>/chris-x509.pem" valid until "2021-02-01 10:33:49 +0000 UTC". client/keystore.go:318
DEBU [CLIENT]    SSH cert not available client/api.go:710
INFO [CLIENT]    no host login given. defaulting to chris client/api.go:1035
INFO [CLIENT]    [KEY AGENT] Connected to the system agent: "/tmp/ssh-XXXXXXr7FKwr/agent.2159" client/api.go:2745
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/home/chris/.tsh/keys/<host>/chris-x509.pem" valid until "2021-02-01 10:33:49 +0000 UTC". client/keystore.go:318
INFO [KEYAGENT]  Loading SSH key for user "chris" and cluster "<host>". client/keyagent.go:167
DEBU [TSH]       SSH cert not available tsh/tsh.go:1776
DEBU [CLIENT]    not using loopback pool for remote proxy addr: <host>:3080 client/api.go:2706
Enter password for Teleport user chris:
DEBU [CLIENT]    not using loopback pool for remote proxy addr: <host>:3080 client/api.go:2706
DEBU [CLIENT]    HTTPS client init(proxyAddr=<host>:3080, insecure=false) client/weblogin.go:211
Tap any security key
DEBU [CLIENT]    prompting U2F devices with facet "https://<host>:3080" client/mfa.go:129

ERROR REPORT:
Original Error: *trace.NotFoundError object not found
Stack Trace:
        github.com/gravitational/teleport/lib/httplib/httplib.go:131 github.com/gravitational/teleport/lib/httplib.ConvertResponse
        github.com/gravitational/teleport/lib/client/https_client.go:76 github.com/gravitational/teleport/lib/client.(*WebClient).PostJSON
        github.com/gravitational/teleport/lib/client/weblogin.go:415 github.com/gravitational/teleport/lib/client.SSHAgentMFALogin
        github.com/gravitational/teleport/lib/client/api.go:2666 github.com/gravitational/teleport/lib/client.(*TeleportClient).mfaLocalLogin
        github.com/gravitational/teleport/lib/client/api.go:2550 github.com/gravitational/teleport/lib/client.(*TeleportClient).localLogin
        github.com/gravitational/teleport/lib/client/api.go:2214 github.com/gravitational/teleport/lib/client.(*TeleportClient).Login
        github.com/gravitational/teleport/tool/tsh/tsh.go:760 main.onLogin
        github.com/gravitational/teleport/tool/tsh/tsh.go:557 main.Run
        github.com/gravitational/teleport/tool/tsh/tsh.go:257 main.main
        runtime/proc.go:225 runtime.main
        runtime/asm_amd64.s:1371 runtime.goexit
User Message: object not found

Local teleport version:

chris@pioneer ~ » teleport version
Teleport v6.2.0 git: go1.16.4
chris@pioneer ~ » pacman -Q teleport
teleport 6.2.1-2

Teleport on proxy:

root@apple ~ # teleport version
Teleport v6.2.1 git:v6.2.1-0-gbada2617a go1.15.5

Teleport on auth:

root@spinach: ~ # teleport version                                                                                                                                                                                     
Teleport v6.2.1 git:v6.2.1-0-gbada2617a go1.15.5

[webvictim]

@awly Could you take a look at this please? It looks like the error may be with MFA.

[awly]

@AritraWork97 @christian-heusel do you see anything interesting in the server log (either proxy or auth)?
Ideally, start proxy and auth in debug mode (with -d flag) and grab the output while performing the login.

Another question - did you recently upgrade teleport or change anything in your teleport.yaml?

[christian-heusel]

You can see the output in the linked video below, IMO there is nothing interesting there ... Instead of failing the Login seems to work 😕
I recently upgraded teleport (via the apt repository) but didnt change anything. The application was relaunched after the upgrade.

teleport.mp4

[awly]

@christian-heusel hmm, that's strange.

• can you paste your teleport.yaml (with anything secret redacted)?
• can you try logging in after changing second_factor: to off in the config?
• do you remember which version you upgraded from?

[christian-heusel]

Setting second_factor: to off didnt help unfortunately 😭
Here are the configs:

ProxyServer:

teleport:
  nodename: apple
  data_dir: /var/lib/teleport
  pid_file: /var/run/teleport.pid
  auth_servers:
  - ip2:3025
  connection_limits:
    max_connections: 1000
    max_users: 250
  log:
    output: stderr
    severity: INFO
auth_service:
  enabled: "no"
ssh_service:
  enabled: "yes"
  listen_addr: 0.0.0.0:3022
proxy_service:
  enabled: "yes"
  listen_addr: 0.0.0.0:3023
  web_listen_addr: 0.0.0.0:3080
  tunnel_listen_addr: 0.0.0.0:3024
  https_key_file: /etc/letsencrypt/live/server2/privkey.pem
  https_cert_file: /etc/letsencrypt/live/server2/fullchain.pem

AuthServer

teleport:
  nodename: spinach
  data_dir: /var/lib/teleport
  pid_file: /var/run/teleport.pid
  auth_servers:
  - ip:3025
  connection_limits:
    max_connections: 1000
    max_users: 250
  log:
    output: stderr
    severity: INFO
auth_service:
  enabled: "yes"
  listen_addr: 0.0.0.0:3025
  cluster_name: "ClusterName"
  authentication:
    type: local
    # second_factor: off
    second_factor: u2f
    u2f:
      app_id: https://server:3080
      facets:
        - https://server:3080
  session_recording: "node"
ssh_service:
  enabled: "yes"
  listen_addr: 0.0.0.0:3022
proxy_service:
  enabled: "no"

[awly]

Sorry for the delay.
Can you try creating a brand new user (tctl users add ...) and logging in with that?
I wonder if something about your user state got corrupted in the storage backend.

[awly]

Also, let's move the debugging to #7117