Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kurtosis does not work on Arch linux #2508

Open
hexash42 opened this issue Jul 2, 2024 · 3 comments
Open

Kurtosis does not work on Arch linux #2508

hexash42 opened this issue Jul 2, 2024 · 3 comments
Labels
bug Something isn't working cli For bugs relating to the CLI critical Critical bug or feature

Comments

@hexash42
Copy link

hexash42 commented Jul 2, 2024

What's your CLI version?

0.90.1

Description & steps to reproduce

Downloaded the amd64 binary on arch linux. Run the quickstart example (Or basically anything else)
kurtosis run github.com/kurtosis-tech/basic-service-package --enclave quickstart

This is the output (hangs forever)

~ ❱ kurtosis run github.com/kurtosis-tech/basic-service-package --enclave quickstart
INFO[2024-07-02T16:28:31+03:00] No Kurtosis engine was found; attempting to start one...
INFO[2024-07-02T16:28:31+03:00] Starting the centralized logs components...
INFO[2024-07-02T16:28:31+03:00] Centralized logs components started.
INFO[2024-07-02T16:28:32+03:00] Reverse proxy started.
INFO[2024-07-02T16:28:33+03:00] Successfully started Kurtosis engine
INFO[2024-07-02T16:28:33+03:00] Creating a new enclave for Starlark to run inside...

This is the engine logs:

�[37mDEBU�[0m[2024-07-02T13:28:33Z][backend_creator.go:getLocalDockerKurtosisBackend] Connecting to Docker daemon via unix socket '/var/run/docker.sock' 
�[37mDEBU�[0m[2024-07-02T13:28:33Z][log_file_manager.go:func1] Scheduling log removal for log retention every '6h0m0s' hours... 
�[37mDEBU�[0m[2024-07-02T13:28:33Z][log_file_manager.go:RemoveLogsBeyondRetentionPeriod] Removed logs beyond retention period at the following path: '/var/log/kurtosis/2024/26/' 
�[37mDEBU�[0m[2024-07-02T13:28:33Z][log_file_manager.go:func2] Scheduling log file path creation every '1m0s' minutes... 
�[36mINFO�[0m[2024-07-02T13:28:33Z][main.go:restApiServer] Running REST API server...                   
�[37mDEBU�[0m[2024-07-02T13:28:33Z][main.go:func1] Created environment js file with content: 
window.env = {}; 
�[36mINFO�[0m[2024-07-02T13:28:33Z][main.go:restApiServer] Setting-up CORS policy to accept requests from origins: [*] 
�[36mINFO�[0m[2024-07-02T13:28:33Z][server.go:RunEnclaveManagerApiServer] Web server running and listening on port 8081 
�[36mINFO�[0m[2024-07-02T13:28:33Z][enclave_rest_api_handler.go:getGrpcClientConn] No API container info is available for enclave a87bf298522047518c5c75b5245f1472 
�[33mWARN�[0m[2024-07-02T13:28:33Z][enclave_rest_api_handler.go:refreshEnclaveConnections] Unavailable gRPC connection to enclave 'a87bf298522047518c5c75b5245f1472', skipping it! 

   ____    __
  / __/___/ /  ___
 / _// __/ _ \/ _ \
/___/\__/_//_/\___/ v4.11.3
High performance, minimalist Go web framework
https://echo.labstack.com
____________________________________O/_______
                                    O\
⇨ http server started on [::]:9779
�[37mDEBU�[0m[2024-07-02T13:28:33Z][engine_connect_server_service.go:CreateEnclave] args: enclave_name:"quickstart" api_container_version_tag:"" api_container_log_level:"debug" mode:TEST should_apic_run_in_debug_mode:false 
�[37mDEBU�[0m[2024-07-02T13:28:33Z][docker_kurtosis_backend_enclave_functions.go:CreateEnclave] Creating Docker network for enclave '6ae3a6dc6b8b413399694b7f3cd5c0b0'... 
�[37mDEBU�[0m[2024-07-02T13:28:33Z][docker_kurtosis_backend_enclave_functions.go:CreateEnclave] Docker network for enclave '6ae3a6dc6b8b413399694b7f3cd5c0b0' created successfully with ID '6a4edea41be43da5753bae2b5c9d4a4009e69fa7c29b01f8875e78e3a231dd1e' 
�[37mDEBU�[0m[2024-07-02T13:28:33Z][shared_helpers.go:getReverseProxyObjectFromContainerInfo] Enclave networks: 'map[5b202c1134cbba6702c4ad3c532963aba47d7100f9689b7eaa93dbe56a7bd212:172.16.0.2]' 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][create_logs_collector.go:CreateLogsCollectorForEnclave] Creating logs collector for enclave '6ae3a6dc6b8b413399694b7f3cd5c0b0' 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][docker_manager.go:FetchImage] Fetching image 'alpine:3.17' with image download mode: missing 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][docker_manager.go:getContainerHostConfig] Binds: [kurtosis-logs-collector-vol--6ae3a6dc6b8b413399694b7f3cd5c0b0:/fluent-bit/etc] 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][docker_manager.go:CreateAndStartContainer] Created container with ID '83272470daa10a6773c8f4879d669f7e2291151d0b5e53f6ae8f75e8ffd11883' from image 'alpine:3.17' 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][fluentbit_configuration_creator.go:createFluentbitConfigFileInVolume] The Fluentbit config file with content '
[SERVICE]
	log_level debug
	http_server On
	http_listen 0.0.0.0
	http_port 9712
	storage.path /fluent-bit/etc/storage/
[INPUT]
	name forward
	listen 0.0.0.0
	port 9713
	storage.type  filesystem
[OUTPUT]
	name forward
	match *
	host 172.17.0.2
	port 9714
' was successfully added into the volume 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][docker_manager.go:FetchImage] Fetching image 'fluent/fluent-bit:1.9.7' with image download mode: missing 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][docker_manager.go:getContainerHostConfig] Binds: [kurtosis-logs-collector-vol--6ae3a6dc6b8b413399694b7f3cd5c0b0:/fluent-bit/etc] 
�[37mDEBU�[0m[2024-07-02T13:28:34Z][docker_manager.go:CreateAndStartContainer] Created container with ID '7c75bcc6ec837b9e2de16c009871efe3f516905dd898ceea2de56bd70b2cf618' from image 'fluent/fluent-bit:1.9.7' 
�[37mDEBU�[0m[2024-07-02T13:28:35Z][create_logs_collector.go:CreateLogsCollectorForEnclave] Checking for logs collector availability in enclave '6ae3a6dc6b8b413399694b7f3cd5c0b0'...

Enclave log from kurtosis-dump:

Fluent Bit v1.9.7
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2024/07/02 13:28:35] [ info] Configuration:
[2024/07/02 13:28:35] [ info]  flush time     | 1.000000 seconds
[2024/07/02 13:28:35] [ info]  grace          | 5 seconds
[2024/07/02 13:28:35] [ info]  daemon         | 0
[2024/07/02 13:28:35] [ info] ___________
[2024/07/02 13:28:35] [ info]  inputs:
[2024/07/02 13:28:35] [ info]      forward
[2024/07/02 13:28:35] [ info] ___________
[2024/07/02 13:28:35] [ info]  filters:
[2024/07/02 13:28:35] [ info] ___________
[2024/07/02 13:28:35] [ info]  outputs:
[2024/07/02 13:28:35] [ info]      forward.0
[2024/07/02 13:28:35] [ info] ___________
[2024/07/02 13:28:35] [ info]  collectors:
[2024/07/02 13:28:35] [ info] [fluent bit] version=1.9.7, commit=265783ebe9, pid=1
[2024/07/02 13:28:35] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2024/07/02 13:28:35] [ info] [storage] created root path /fluent-bit/etc/storage/
[2024/07/02 13:28:35] [ info] [storage] version=1.2.0, type=memory+filesystem, sync=normal, checksum=disabled, max_chunks_up=128
[2024/07/02 13:28:35] [ info] [storage] backlog input plugin: storage_backlog.1
[2024/07/02 13:28:35] [ info] [cmetrics] version=0.3.5
[2024/07/02 13:28:35] [debug] [forward:forward.0] created event channels: read=21 write=22
[2024/07/02 13:28:35] [debug] [in_fw] Listen='0.0.0.0' TCP_Port=9713
[2024/07/02 13:28:35] [ info] [input:forward:forward.0] listening on 0.0.0.0:9713
[2024/07/02 13:28:35] [debug] [storage_backlog:storage_backlog.1] created event channels: read=24 write=25
[2024/07/02 13:28:35] [ info] [input:storage_backlog:storage_backlog.1] queue memory limit: 95.4M
[2024/07/02 13:28:35] [debug] [forward:forward.0] created event channels: read=26 write=27
[2024/07/02 13:28:35] [debug] [router] match rule forward.0:forward.0
[2024/07/02 13:28:35] [debug] [router] match rule storage_backlog.1:forward.0
[2024/07/02 13:28:35] [ info] [output:forward:forward.0] worker #0 started
[2024/07/02 13:28:35] [ info] [output:forward:forward.0] worker #1 started
[2024/07/02 13:28:35] [ info] [http_server] listen iface=0.0.0.0 tcp_port=9712
[2024/07/02 13:28:35] [ info] [sp] stream processor started

The engine is 100% stuck and does not respond to stuff like kurtosis enclave ls - it must first be restarted

Desired behavior

I expect this to work just like the quickstart example

What is the severity of this bug?

Critical; I am blocked and Kurtosis is unusable for me because of this bug.

What area of the product does this pertain to?

CLI: the Command Line Interface
@hexash42 hexash42 added the bug Something isn't working label Jul 2, 2024
@github-actions github-actions bot added cli For bugs relating to the CLI critical Critical bug or feature labels Jul 2, 2024
@skylarmb
Copy link
Contributor

skylarmb commented Jul 2, 2024

what is the status of the logs aggregator docker container (the image it runs is timberio/vector). I think i ran into something like this before on fedora, but not sure it was the same issue:

image

@hexash42
Copy link
Author

hexash42 commented Jul 3, 2024

Looks like it is up

846a688ecd4f   fluent/fluent-bit:1.9.7         "/fluent-bit/bin/flu…"   37 seconds ago   Up 36 seconds   2020/tcp                                                                           kurtosis-logs-collector--0ce18db33a5d4a28b6216331c96a684f

And here are the logs from docker log

Fluent Bit v1.9.7
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2024/07/03 14:56:10] [ info] Configuration:
[2024/07/03 14:56:10] [ info]  flush time     | 1.000000 seconds
[2024/07/03 14:56:10] [ info]  grace          | 5 seconds
[2024/07/03 14:56:10] [ info]  daemon         | 0
[2024/07/03 14:56:10] [ info] ___________
[2024/07/03 14:56:10] [ info]  inputs:
[2024/07/03 14:56:10] [ info]      forward
[2024/07/03 14:56:10] [ info] ___________
[2024/07/03 14:56:10] [ info]  filters:
[2024/07/03 14:56:10] [ info] ___________
[2024/07/03 14:56:10] [ info]  outputs:
[2024/07/03 14:56:10] [ info]      forward.0
[2024/07/03 14:56:10] [ info] ___________
[2024/07/03 14:56:10] [ info]  collectors:
[2024/07/03 14:56:10] [ info] [fluent bit] version=1.9.7, commit=265783ebe9, pid=1
[2024/07/03 14:56:10] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2024/07/03 14:56:10] [ info] [storage] created root path /fluent-bit/etc/storage/
[2024/07/03 14:56:10] [ info] [storage] version=1.2.0, type=memory+filesystem, sync=normal, checksum=disabled, max_chunks_up=128
[2024/07/03 14:56:10] [ info] [storage] backlog input plugin: storage_backlog.1
[2024/07/03 14:56:10] [ info] [cmetrics] version=0.3.5
[2024/07/03 14:56:10] [debug] [forward:forward.0] created event channels: read=21 write=22
[2024/07/03 14:56:10] [debug] [in_fw] Listen='0.0.0.0' TCP_Port=9713
[2024/07/03 14:56:10] [ info] [input:forward:forward.0] listening on 0.0.0.0:9713
[2024/07/03 14:56:10] [debug] [storage_backlog:storage_backlog.1] created event channels: read=24 write=25
[2024/07/03 14:56:10] [ info] [input:storage_backlog:storage_backlog.1] queue memory limit: 95.4M
[2024/07/03 14:56:10] [debug] [forward:forward.0] created event channels: read=26 write=27
[2024/07/03 14:56:10] [debug] [router] match rule forward.0:forward.0
[2024/07/03 14:56:10] [debug] [router] match rule storage_backlog.1:forward.0
[2024/07/03 14:56:10] [ info] [output:forward:forward.0] worker #0 started
[2024/07/03 14:56:10] [ info] [output:forward:forward.0] worker #1 started
[2024/07/03 14:56:10] [ info] [http_server] listen iface=0.0.0.0 tcp_port=9712
[2024/07/03 14:56:10] [ info] [sp] stream processor started

@skylarmb
Copy link
Contributor

skylarmb commented Jul 8, 2024 •
edited
Loading

yeah nvm then, this has similar symptoms to but is not the same issue I experienced on aarch64 Fedora, must be something else. I have an amd64 Manjaro machine I will test on later today

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working cli For bugs relating to the CLI critical Critical bug or feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@skylarmb @hexash42