Describe the bug
So im seeing this weird issue with loki after upgrading from the helm chart 2.12.2 to 3.4.2. Our current setup stores logs in our file system and I want to upgrade to an equivalant set up in 3.4.2, which i assume is the single binary/single tenant mode.
I can’t query old log messages despite pointing the new instance loki at the old directory that the older version of loki was using. Querying new log data still works. Querying also works if i downgrade to 2.x, I’ve also tried upgrading to the latest 2.x and then to 3.x and it yields the same result
This is the error i get when trying to query older data:
open /data/loki/chunks/fake/6c8c778b0292b2a0/MTg0ODdjM2RjNTM6MTg0ODgzMjI1M2Q6MTJmNGNmZTM=: no such file or directory
It appears that the way chunks are being stored has changed because the structure of our history logs aren’t paritioned by using a tenant id called “fake” and there’s not a directory with an alpha-numeric string like “6c8c778b0292b2a0” within the path of our historical chunk files.
Will it be possible to retain the ability to query our historical logs from the 2.x when upgrading to 3.x?
To Reproduce
Steps to reproduce the behavior:
- Started Loki (2.6.1) helm : 2.12.2
- Started Promtail (2.6.1) helm: 6.6.2
- Query: run any query that would work on a cluster running 2.X then migrate to 3.4.2 and run that same query
Expected behavior
I expect to be able to query all data collected by promtail irrespective of the loki helm chart version
Environment:
- Infrastructure: EKS, kubenetes - 1.22
- Deployment tool: helm