Flushing stream

Last week I upgraded to version 3.0.0 of Loki, and I see a lot of log entries with the message msg="flushing stream"

Apr 14 14:37:30 srv2 loki[175740]: level=info ts=2024-04-14T14:37:30.781757363Z caller=flush.go:176 component=ingester msg="flushing stream" user=fake fp=e399213816304d08 immediate=false num_chunks=1 labels="{application=\"dhcpd\", co
nnection_hostname=\"xxxx.xxxx.xx\", facility=\"local7\", host=\"xxxx.xxxx.xx\", hostname=\"xxxx.xxxx.xx\", job=\"syslog\", level=\"debug\", service_name=\"dhcpd\"}"

I cannot find the log entries, so they are not stored in storage. I guess this is a problem, how can I solve this?

Thanks,

B

What else changed besides the version upgrade? Double check your storage and see if chunks show up there. Also if your ingester can’t write to storage there must be some logs with the reason.

Thanks @tonyswumac, Chunks show up:

x@y /var/lib/loki: find ./chunks -type f | wc -l
26388

I can not find any write errors in the logs.

I changed the configuration to the distributed config and deleted all the storage.

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  instance_addr: 127.0.0.1
  path_prefix: /var/lib/loki
  storage:
    filesystem:
      chunks_directory: /var/lib/loki/chunks
      rules_directory: /var/lib/loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

query_range:
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        max_size_mb: 100

schema_config:
  configs:
    - from: 2020-10-24
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  max_query_lookback: 672h # 28 days
  retention_period: 672h   # 28 days

compactor:
  working_directory: /var/lib/loki/rules/retention
  compaction_interval: 10m
  retention_enabled: true
  retention_delete_delay: 2h
  delete_request_store: filesystem

ruler:
  alertmanager_url: http://localhost:9093

Many thanks!