403 Forbidden & Cannot Login To Web UI & Some WARN

[maginati0n]

Hello, let me ask a question on this discussion.
I installed teleport using Kubernetes via helm, for one week no problems until today, I joined more than 150 servers into the teleport server. But when I want to check if all servers have been input into the teleport server, I can't access the teleport web UI.
There is an Error 403 ( Forbidden ) and an access denied error message as shown below

And when I open the logs on the Kubernetes pods on the lens, there is a warning like a picture below

I hope what I say can be conveyed clearly, because I am not fluent in English, and this is my first discussion on Github.
Thank you

[maginati0n]

WARN [SQLITE] "SLOW TRANSACTION: 1.223915937s, goroutine 3598 [running]:\nruntime/debug.Stack(0xc02e7c824dbb85db, 0x492b4fb1d84, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c824dbb85db, 0x492b4fb1d84, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc00519fec0, 0xc004963268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc00519fec0, 0xc002a9ab40, 0x33, 0x40, 0x0, 0xda2363c, 0xed86b73e1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc00519fec0, 0xc002a9ab40, 0x33, 0x40, 0x0, 0xda2363c, 0xed86b73e1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc00519fec0, 0xc002a9ab40, 0x33, 0x40, 0x0, 0xda2363c, 0xed86b73e1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc00519fec0, 0xc0057b3920, 0x24, 0x2d7dd34, 0x7, 0x0, 0xda2363c, 0xed86b73e1, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc00033cbd0, 0x31ed9c0, 0xc00519fec0, 0xc0057b3920, 0x24, 0x0, 0x0, 0x0, 0xda2363c, 0xed86b73e1, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc0017866d0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc000d48540, 0x2be3180, 0xc0051a1200)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc0040e4240, 0xc000d484a0, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc00273d680, 0xc0039dfa00, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc00273d680, 0xc0039dfa00, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc003eb9140, 0xc000d3ba00, 0x3210d20, 0xc00273d680, 0xc0039dfa00)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.18026781s, goroutine 383338 [running]:\nruntime/debug.Stack(0xc02e7c83716e4efd, 0x493c7190e96, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c83716e4efd, 0x493c7190e96, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc006690600, 0xc0050d12d0, 0x317fba0, 0xc0067d06c0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).Get(0xc0004a64e0, 0x31ed9c0, 0xc006690600, 0xc001d2a460, 0x4e, 0x50, 0xc000110900, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:569 +0xf6\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).Get(0xc000193b10, 0x31ed9c0, 0xc006690600, 0xc001d2a460, 0x4e, 0x50, 0x50, 0xc001d2a460, 0xc0050d1410)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:97 +0xa8\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).Get(0xc000c09a40, 0x31ed9c0, 0xc006690600, 0xc001d2a460, 0x4e, 0x50, 0x4e, 0x4, 0x4)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:161 +0x253\ngithub.com/gravitational/teleport/lib/services/local.(*webSessions).Get(0xc004b7ce20, 0x31ed9c0, 0xc006690600, 0xc002ce55a0, 0x9, 0xc00308d640, 0x40, 0x0, 0x0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/session.go:111 +0x21a\ngithub.com/gravitational/teleport/lib/auth.(*webSessionsWithRoles).Get(0xc004b7ce40, 0x31ed9c0, 0xc006690600, 0xc002ce55a0, 0x9, 0xc00308d640, 0x40, 0x0, 0x0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:864 +0xce\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).GetWebSession(0xc000204990, 0x31ed9c0, 0xc006690600, 0xc003e459c0, 0xc000204990, 0x284a920, 0x312dea0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:1138 +0x1ad\ngithub.com/gravitational/teleport/api/client/proto._AuthService_GetWebSession_Handler.func1(0x31ed9c0, 0xc006690600, 0x2c11360, 0xc003e459c0, 0x2be3180, 0xc00526a000, 0x31ed9c0, 0xc006690600)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:9696 +0x8b\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).UnaryInterceptor(0xc000a8af40, 0x31ed9c0, 0xc006690510, 0x2c11360, 0xc003e459c0, 0xc004b7ca60, 0xc004b7ca80, 0x0, 0x0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:355 +0x614\ngithub.com/gravitational/teleport/api/client/proto._AuthService_GetWebSession_Handler(0x2d6c8a0, 0xc000204990, 0x31ed9c0, 0xc0066904e0, 0xc006d93260, 0xc000ab5fb0, 0x31ed9c0, 0xc0066904e0, 0xc001d2a3c0, 0x4d)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:9698 +0x150\ngoogle.golang.org/grpc.(*Server).processUnaryRPC(0xc000d3ba00, 0x3210d20, 0xc000e6c000, 0xc0038ce600, 0xc0004993e0, 0x471db70, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1082 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc000e6c000, 0xc0038ce600, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1405 +0xcc5\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000305040, 0xc000d3ba00, 0x3210d20, 0xc000e6c000, 0xc0038ce600)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.214842938s, goroutine 1891 [running]:\nruntime/debug.Stack(0xc02e7c8611f8e65b, 0x496374a53f0, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c8611f8e65b, 0x496374a53f0, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc000f14c60, 0xc003059268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc000f14c60, 0xc00105d5c0, 0x33, 0x40, 0x0, 0x11b77eb3, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc000f14c60, 0xc00105d5c0, 0x33, 0x40, 0x0, 0x11b77eb3, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc000f14c60, 0xc00105d5c0, 0x33, 0x40, 0x0, 0x11b77eb3, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc000f14c60, 0xc0006ebd40, 0x24, 0x2d7dd34, 0x7, 0x0, 0x11b77eb3, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc000935730, 0x31ed9c0, 0xc000f14c60, 0xc0006ebd40, 0x24, 0x0, 0x0, 0x0, 0x11b77eb3, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc000a78680, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc0026d7020, 0x2be3180, 0xc0027e9200)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc001a7c540, 0xc0026d6fe0, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc00198a300, 0xc0001f5400, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc00198a300, 0xc0001f5400, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0009fa9b0, 0xc000d3ba00, 0x3210d20, 0xc00198a300, 0xc0001f5400)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.396781071s, goroutine 5241 [running]:\nruntime/debug.Stack(0xc02e7c861075975c, 0x49635c704f1, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c861075975c, 0x49635c704f1, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc006ac3ec0, 0xc0018c7268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc006ac3ec0, 0xc003f62700, 0x33, 0x40, 0x0, 0x12af85c0, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc006ac3ec0, 0xc003f62700, 0x33, 0x40, 0x0, 0x12af85c0, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc006ac3ec0, 0xc003f62700, 0x33, 0x40, 0x0, 0x12af85c0, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc006ac3ec0, 0xc00497cde0, 0x24, 0x2d7dd34, 0x7, 0x0, 0x12af85c0, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc000934bd0, 0x31ed9c0, 0xc006ac3ec0, 0xc00497cde0, 0x24, 0x0, 0x0, 0x0, 0x12af85c0, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc006acf6e0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc0012f1e60, 0x2be3180, 0xc006ae6400)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc000e4f740, 0xc0012f1e20, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc001d37980, 0xc0033d4e00, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc001d37980, 0xc0033d4e00, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc003fddc30, 0xc000d3ba00, 0x3210d20, 0xc001d37980, 0xc0033d4e00)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.588331629s, goroutine 5506 [running]:\nruntime/debug.Stack(0xc02e7c860e9e0008, 0x49633ef6d9d, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c860e9e0008, 0x49633ef6d9d, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc006b1e870, 0xc006c43268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc006b1e870, 0xc00303f3c0, 0x33, 0x40, 0x0, 0xe26d605, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc006b1e870, 0xc00303f3c0, 0x33, 0x40, 0x0, 0xe26d605, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc006b1e870, 0xc00303f3c0, 0x33, 0x40, 0x0, 0xe26d605, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc006b1e870, 0xc003e4bda0, 0x24, 0x2d7dd34, 0x7, 0x0, 0xe26d605, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc003d861c0, 0x31ed9c0, 0xc006b1e870, 0xc003e4bda0, 0x24, 0x0, 0x0, 0x0, 0xe26d605, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc001b7ce60, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc00049cf60, 0x2be3180, 0xc004439400)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc004064180, 0xc00049cec0, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc001400600, 0xc003036d00, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc001400600, 0xc003036d00, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc00113cb70, 0xc000d3ba00, 0x3210d20, 0xc001400600, 0xc003036d00)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.644737221s, goroutine 5979 [running]:\nruntime/debug.Stack(0xc02e7c860ed25348, 0x4963423c0d8, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c860ed25348, 0x4963423c0d8, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc003180630, 0xc006c8b268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc003180630, 0xc00303f840, 0x33, 0x40, 0x0, 0xec92b60, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc003180630, 0xc00303f840, 0x33, 0x40, 0x0, 0xec92b60, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc003180630, 0xc00303f840, 0x33, 0x40, 0x0, 0xec92b60, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc003180630, 0xc003e4be00, 0x24, 0x2d7dd34, 0x7, 0x0, 0xec92b60, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc0020e07e0, 0x31ed9c0, 0xc003180630, 0xc003e4be00, 0x24, 0x0, 0x0, 0x0, 0xec92b60, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc00467c670, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc0040d29e0, 0x2be3180, 0xc004500a00)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc000e4e540, 0xc0040d29a0, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc002dd0c00, 0xc00368d500, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc002dd0c00, 0xc00368d500, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000f2d1c0, 0xc000d3ba00, 0x3210d20, 0xc002dd0c00, 0xc00368d500)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.772394629s, goroutine 4830 [running]:\nruntime/debug.Stack(0xc02e7c86162092b6, 0x4963b72004a, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c86162092b6, 0x4963b72004a, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc005336d20, 0xc00382b268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc005336d20, 0xc004487240, 0x33, 0x40, 0x0, 0x15d2b04d, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc005336d20, 0xc004487240, 0x33, 0x40, 0x0, 0x15d2b04d, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc005336d20, 0xc004487240, 0x33, 0x40, 0x0, 0x15d2b04d, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc005336d20, 0xc0042d53b0, 0x24, 0x2d7dd34, 0x7, 0x0, 0x15d2b04d, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc003d86070, 0x31ed9c0, 0xc005336d20, 0xc0042d53b0, 0x24, 0x0, 0x0, 0x0, 0x15d2b04d, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc00414d610, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc0043a9ce0, 0x2be3180, 0xc005403600)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc004065680, 0xc0043a9ca0, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc003562780, 0xc0038cfb00, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc003562780, 0xc0038cfb00, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc003f37f10, 0xc000d3ba00, 0x3210d20, 0xc003562780, 0xc0038cfb00)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.479506118s, goroutine 3156 [running]:\nruntime/debug.Stack(0xc02e7c863156a83d, 0x49656a815e4, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c863156a83d, 0x49656a815e4, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc004add9e0, 0xc005bc7268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc004add9e0, 0xc00105d9c0, 0x33, 0x40, 0x0, 0x312d886c, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc004add9e0, 0xc00105d9c0, 0x33, 0x40, 0x0, 0x312d886c, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc004add9e0, 0xc00105d9c0, 0x33, 0x40, 0x0, 0x312d886c, 0xed86b73f0, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc004add9e0, 0xc0039f80c0, 0x24, 0x2d7dd34, 0x7, 0x0, 0x312d886c, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc003d86b60, 0x31ed9c0, 0xc004add9e0, 0xc0039f80c0, 0x24, 0x0, 0x0, 0x0, 0x312d886c, 0xed86b73f0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc004898660, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc000cb6520, 0x2be3180, 0xc001a6ca00)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc000225bc0, 0xc000cb64e0, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc001d37200, 0xc0039de200, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc001d37200, 0xc0039de200, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc004b54570, 0xc000d3ba00, 0x3210d20, 0xc001d37200, 0xc0039de200)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.254780485s, goroutine 3750 [running]:\nruntime/debug.Stack(0xc02e7c865957bcb5, 0x4967a43f44c, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c865957bcb5, 0x4967a43f44c, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc005674270, 0xc005a6b268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc005674270, 0xc0047821c0, 0x33, 0x40, 0x0, 0x193aada0, 0xed86b73f1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc005674270, 0xc0047821c0, 0x33, 0x40, 0x0, 0x193aada0, 0xed86b73f1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc005674270, 0xc0047821c0, 0x33, 0x40, 0x0, 0x193aada0, 0xed86b73f1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc005674270, 0xc000a90420, 0x24, 0x2d7dd34, 0x7, 0x0, 0x193aada0, 0xed86b73f1, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc00033d880, 0x31ed9c0, 0xc005674270, 0xc000a90420, 0x24, 0x0, 0x0, 0x0, 0x193aada0, 0xed86b73f1, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc0019d3ea0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc001eeed60, 0x2be3180, 0xc00566c200)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc0040e49c0, 0xc001eeed20, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc00486be00, 0xc00034dc00, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc00486be00, 0xc00034dc00, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc001f24ad0, 0xc000d3ba00, 0x3210d20, 0xc00486be00, 0xc00034dc00)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817 WARN [SQLITE] "SLOW TRANSACTION: 1.320038964s, goroutine 6304 [running]:\nruntime/debug.Stack(0xc02e7c867b031d85, 0x4969bef5518, 0x47db200)\n\t/opt/go/src/runtime/debug/stack.go:24 +0x9f\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction.func1(0xc02e7c867b031d85, 0x4969bef5518, 0x47db200, 0xc0004a64e0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:817 +0x93\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).inTransaction(0xc0004a64e0, 0x31ed9c0, 0xc0059b6780, 0xc003b51268, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:868 +0x2ea\ngithub.com/gravitational/teleport/lib/backend/lite.(*Backend).KeepAlive(0xc0004a64e0, 0x31ed9c0, 0xc0059b6780, 0xc0005ad000, 0x33, 0x40, 0x0, 0x3ac5d275, 0xed86b73f1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/lite/lite.go:659 +0x1c9\ngithub.com/gravitational/teleport/lib/backend.(*Sanitizer).KeepAlive(0xc000193b10, 0x31ed9c0, 0xc0059b6780, 0xc0005ad000, 0x33, 0x40, 0x0, 0x3ac5d275, 0xed86b73f1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/sanitize.go:137 +0xe5\ngithub.com/gravitational/teleport/lib/backend.(*Reporter).KeepAlive(0xc000c09a40, 0x31ed9c0, 0xc0059b6780, 0xc0005ad000, 0x33, 0x40, 0x0, 0x3ac5d275, 0xed86b73f1, 0x0, ...)\n\t/go/src/github.com/gravitational/teleport/lib/backend/report.go:215 +0x10b\ngithub.com/gravitational/teleport/lib/services/local.(*PresenceService).KeepAliveServer(0xc0001b06c0, 0x31ed9c0, 0xc0059b6780, 0xc0042d5ef0, 0x24, 0x2d7dd34, 0x7, 0x0, 0x3ac5d275, 0xed86b73f1, ...)\n\t/go/src/github.com/gravitational/teleport/lib/services/local/presence.go:1215 +0x3ac\ngithub.com/gravitational/teleport/lib/auth.(*ServerWithRoles).KeepAliveServer(0xc000396c40, 0x31ed9c0, 0xc0059b6780, 0xc0042d5ef0, 0x24, 0x0, 0x0, 0x0, 0x3ac5d275, 0xed86b73f1, ...)\n\t/go/src/github.com/gravitational/teleport/lib/auth/auth_with_roles.go:473 +0x2db\ngithub.com/gravitational/teleport/lib/auth.(*GRPCServer).SendKeepAlives(0xc000204990, 0x320c280, 0xc0059ab170, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/grpcserver.go:124 +0x2b8\ngithub.com/gravitational/teleport/api/client/proto._AuthService_SendKeepAlives_Handler(0x2d6c8a0, 0xc000204990, 0x3201600, 0xc001b861c0, 0x2be3180, 0xc0055e9a00)\n\t/go/src/github.com/gravitational/teleport/vendor/github.com/gravitational/teleport/api/client/proto/authservice.pb.go:8880 +0xb0\ngithub.com/gravitational/teleport/lib/auth.(*Middleware).StreamInterceptor(0xc000a8af40, 0x2d6c8a0, 0xc000204990, 0x3201d80, 0xc0007c0c00, 0xc001b86120, 0x2eb2a50, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/lib/auth/middleware.go:386 +0x635\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc000d3ba00, 0x3210d20, 0xc002dd1080, 0xc003a51400, 0xc0004993e0, 0x4716b60, 0x0, 0x0, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1336 +0x522\ngoogle.golang.org/grpc.(*Server).handleStream(0xc000d3ba00, 0x3210d20, 0xc002dd1080, 0xc003a51400, 0x0)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:1409 +0xc5c\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0041b0800, 0xc000d3ba00, 0x3210d20, 0xc002dd1080, 0xc003a51400)\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:746 +0xa5\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/go/src/github.com/gravitational/teleport/vendor/google.golang.org/grpc/server.go:744 +0xa5\n." lite/lite.go:817

[stevenGravy]

Hi @maginati0n what is the storage mechanism for k8s here? It looks like the storage may be too slow. Teleport requires SSD storage or one of our HA configurations, not a network drive if that's what's being used.

[maginati0n]

I'm using FreeNAS with Seagate iron wolf hard drive.... so i have to use HA ? or can not single cluster?

[webvictim]

Your original screenshot looks like your username/password/2-factor combination is incorrect so the server is denying you access. You can reset your password by running tctl users reset <username> on the Teleport auth server and following the link.