Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

error running loki #15851

Closed
harsh0522 opened this issue Jan 21, 2025 · 3 comments
Closed

error running loki #15851

harsh0522 opened this issue Jan 21, 2025 · 3 comments

Comments

@harsh0522
Copy link

harsh0522 commented Jan 21, 2025 •
edited
Loading

PLEASE HELP

I am getting this error
Amazon EKS 1.29

Installation values.yaml file

loki:
  auth_enabled: false
  commonConfig:
    replication_factor: 1
  fullnameOverride: loki
  config:
    table_manager:
      retention_deletes_enabled: true
      retention_period: 15d
    chunk_store_config:
      max_look_back_period: 15d
  persistence:
    enabled: true
    size: 50Gi
memberlist:
  service.publishNotReadyAddresses: true
monitoring:
  lokiCanary:
    enabled: false
  selfMonitoring:
    enabled: false
    grafanaAgent:
      installOperator: false
  serviceMonitor:
    metricsInstance:
      enabled: false
gateway:
  enabled: false
test:
  enabled: false
singleBinary:
  replicas: 1

command ---
1- helm repo add grafana https://grafana.github.io/helm-charts

2- helm upgrade --install loki grafana/loki-stack --set grafana.enabled=false,prometheus.enabled=false,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false --set persistence.storageClassName="gp2" --set persistence.enabled=true --set adminPassword='abc' -f values.yaml --set service.type=LoadBalancer --namespace demo

ERROR****

level=warn ts=2025-01-21T04:19:19.986313197Z caller=store.go:51 msg="running with DEPRECATED flag -store.max-look-back-period, use -querier.max-query-lookback instead."
level=info ts=2025-01-21T04:19:19.987290384Z caller=main.go:103 msg="Starting Loki" version="(version=2.6.1, branch=HEAD, revision=6bd05c9a4)"
level=info ts=2025-01-21T04:19:19.987564625Z caller=server.go:288 http=[::]:3100 grpc=[::]:9095 msg="server listening on addresses"
level=info ts=2025-01-21T04:19:19.987945027Z caller=modules.go:736 msg="RulerStorage is not configured in single binary mode and will not be started."
level=warn ts=2025-01-21T04:19:19.98883428Z caller=experimental.go:20 msg="experimental feature in use" feature="In-memory (FIFO) cache - chunksfifocache"
level=info ts=2025-01-21T04:19:19.989368465Z caller=table_manager.go:252 msg="query readiness setup completed" duration=1.608µs distinct_users_len=0
level=info ts=2025-01-21T04:19:19.989865048Z caller=shipper.go:124 msg="starting index shipper in RW mode"
level=info ts=2025-01-21T04:19:19.989983872Z caller=table_manager.go:134 msg="uploading tables"
level=info ts=2025-01-21T04:19:19.990248075Z caller=shipper_index_client.go:79 msg="starting boltdb shipper in RW mode"
level=info ts=2025-01-21T04:19:19.990432621Z caller=table_manager.go:167 msg="handing over indexes to shipper"
level=info ts=2025-01-21T04:19:19.992863041Z caller=modules.go:761 msg="RulerStorage is nil. Not starting the ruler."
level=info ts=2025-01-21T04:19:19.993117317Z caller=worker.go:112 msg="Starting querier worker using query-scheduler and scheduler ring for addresses"
level=info ts=2025-01-21T04:19:19.994932476Z caller=module_service.go:82 msg=initialising module=server
level=info ts=2025-01-21T04:19:19.99497552Z caller=module_service.go:82 msg=initialising module=memberlist-kv
level=info ts=2025-01-21T04:19:19.995020007Z caller=module_service.go:82 msg=initialising module=query-frontend-tripperware
level=error ts=2025-01-21T04:19:19.995120849Z caller=loki.go:386 msg="module failed" module=memberlist-kv error="invalid service state: Stopping, expected: Running"
level=error ts=2025-01-21T04:19:19.995149807Z caller=loki.go:386 msg="module failed" module=ingester-querier error="failed to start ingester-querier, because it depends on module memberlist-kv, which has failed: context canceled"
level=error ts=2025-01-21T04:19:19.995163521Z caller=loki.go:386 msg="module failed" module=query-frontend error="failed to start query-frontend, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Stopping, expected: Running"
level=error ts=2025-01-21T04:19:19.995173124Z caller=loki.go:386 msg="module failed" module=store error="failed to start store, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Stopping, expected: Running"
level=error ts=2025-01-21T04:19:19.995188795Z caller=loki.go:386 msg="module failed" module=usage-report error="failed to start usage-report, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Stopping, expected: Running"
level=info ts=2025-01-21T04:19:19.995187271Z caller=module_service.go:114 msg="module stopped" module=query-frontend-tripperware
level=error ts=2025-01-21T04:19:19.99520084Z caller=loki.go:386 msg="module failed" module=distributor error="failed to start distributor, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Stopping, expected: Running"
level=error ts=2025-01-21T04:19:19.995226816Z caller=loki.go:386 msg="module failed" module=ring error="failed to start ring, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Stopping, expected: Running"
level=error ts=2025-01-21T04:19:19.995246732Z caller=loki.go:386 msg="module failed" module=compactor error="failed to start compactor, because it depends on module usage-report, which has failed: context canceled"
level=error ts=2025-01-21T04:19:19.995257889Z caller=loki.go:386 msg="module failed" module=querier error="failed to start querier, because it depends on module store, which has failed: context canceled"
level=error ts=2025-01-21T04:19:19.995271658Z caller=loki.go:386 msg="module failed" module=query-scheduler error="failed to start query-scheduler, because it depends on module ring, which has failed: context canceled"
level=error ts=2025-01-21T04:19:19.995284919Z caller=loki.go:386 msg="module failed" module=ingester error="failed to start ingester, because it depends on module store, which has failed: context canceled"
level=info ts=2025-01-21T04:19:19.99645112Z caller=modules.go:1090 msg="server stopped"
level=info ts=2025-01-21T04:19:19.99648449Z caller=module_service.go:114 msg="module stopped" module=server
level=info ts=2025-01-21T04:19:19.996522198Z caller=loki.go:375 msg="Loki stopped"
failed services
github.com/grafana/loki/pkg/loki.(*Loki).Run
/src/loki/pkg/loki/loki.go:421
main.main
/src/loki/cmd/loki/main.go:105
runtime.main
/usr/local/go/src/runtime/proc.go:255
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1581
level=error ts=2025-01-21T04:19:19.996602705Z caller=log.go:103 msg="error running loki" err="failed services\ngithub.com/grafana/loki/pkg/loki.(*Loki).Run\n\t/src/loki/pkg/loki/loki.go:421\nmain.main\n\t/src/loki/cmd/loki/main.go:105\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1581"
@JStickler
Copy link
Contributor

The loki-stack Helm chart is deprecated. The Grafana supported Loki Helm chart is the loki chart.

running with DEPRECATED flag -store.max-look-back-period, use -querier.max-query-lookback instead

You need to replace -store.max-look-back-period, which is deprecated, with -querier.max-query-lookback

This is documented in the Upgrade Guide.

@harsh0522
Copy link
Author

Image

it shows me this error after replacing values....

installation command
helm upgrade --install loki grafana/loki-stack --set grafana.enabled=false,prometheus.enabled=false,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false --set persistence.storageClassName="gp2" --set persistence.enabled=true --set adminPassword='abc' -f values.yaml --set service.type=LoadBalancer --namespace demo

should i install loki grafana/loki ????

doest it have a promtail too

@JStickler
Copy link
Contributor

I feel like there's a lot of information missing here.
Which version of Loki are you on, and which version of Loki are you trying to upgrade to? It's hard to provide any help without knowing which Loki version you're working with.

As I said in my last comment, the loki-distributed Helm chart is not supported. We've documented how to migrate to the loki Helm chart.
https://grafana.com/docs/loki/latest/setup/migrate/migrate-from-distributed/

I see in your values. yaml file that you're configuring the TableManager, but you don't need the TableManager if you're using BoltDB or TSDB. And TSDB is the recommended index store for Loki 2.8 and newer. We've documented how to migrate to TSDB here https://grafana.com/docs/loki/latest/setup/migrate/migrate-to-tsdb/ .

Promtail is no longer the recommended client for Loki, we're recommending people switch to Grafana Alloy.
We've documented how to migrate to Alloy here
https://grafana.com/docs/loki/latest/setup/migrate/migrate-to-alloy/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JStickler @harsh0522