Hi,
I have multitenance set. I have been testing data scraping for one of the tenants and created a lot of unwanted data which are preventing loading actual data.
These data are for 90 days and I need to properly place them to the series.
I have loki 3.6, using tsdb and azure storage.
I have tried deleting index data of the tenant and whole tenant folder from blob storage and from tsdb_shipper_cache, deleted postions on promtail agent and still it doesn’t want to load data back to tenant.
When I have redirected promtail to different tenant it collected data properly and displayed them in grafana.
Is there any posibility how to totally delete tenant from loki totally?
I don’t think your issue has anything to do with whatever data you deleted for the tenant. If you are able to send data to Loki, then it’s probably working fine.
What errors do you see on Loki?
What errors do you see on your log agent (sounds like you are using promtail?)
I am getting 404 from azure blob doesn’t exist like this:
Feb 5 14:26:15 vm-XXX-monitoring loki[410801]: level=error ts=2026-02-05T13:26:15.65498508Z caller=parallel_chunk_fetch.go:74 msg="error fetching chunks" err="failed to load chunk 'productionstats-XXX/98c52fcfc020cd3f/19c1b6f3980:19c1b6f3980:6582f5d8': -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /src/loki/vendor/github.com/Azure/azure-storage-blob-go/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=BlobNotFound) =====\nDescription=The specified blob does not exist.\nRequestId:c7e25ebe-e01e-0049-3da3-9606fd000000\nTime:2026-02-05T13:26:15.6553425Z, Details: \n Code: BlobNotFound\n GET https://saYYY.blob.core.windows.net/loki-prod/productionstats-XXX/98c52fcfc020cd3f/19c1b6f3980-19c1b6f3980-6582f5d8?timeout=31\n Authorization: REDACTED\n User-Agent: [Azure-Storage/0.14 (go1.24.10; linux)]\n X-Ms-Client-Request-Id: [e92e196c-5dce-4af6-7eef-49f45ca9ad0e]\n X-Ms-Version: [2020-04-08]\n --------------------------------------------------------------------------------\n RESPONSE Status: 404 The specified blob does not exist.\n Content-Length: [215]\n Content-Type: [application/xml]\n Date: [Thu, 05 Feb 2026 13:26:15 GMT]\n Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]\n X-Ms-Client-Request-Id: [e92e196c-5dce-4af6-7eef-49f45ca9ad0e]\n X-Ms-Error-Code: [BlobNotFound]\n X-Ms-Request-Id: [c7e25ebe-e01e-0049-3da3-9606fd000000]\n X-Ms-Version: [2020-04-08]\n\n\n"
I can see that tenant is recreated in the storage but it seems that index is still old. I have been waiting to WALs been shipped out from loki local storage then doing stop of promtails and loki, cleaning up and starting things again from scrath but same result.
when I try search in grafana it definitely get new data up as it can recognize labels and it starts going through data, but it is just refering old non-existing blobs:
Ok, now I am confused on what exactly you are asking for. From your posts, I understand that:
You’ve removed old chunk files for this one tenant.
New logs are coming in correctly.
When querying logs against the tenant you get errors where certain chunks cannot be located. This is because you removed some chunk files, so this is expected.
So if new logs are coming in and old logs are gone, sounds that’s what you were looking for. What exactly are you looking to solve?
I’ve removed whole tenant folder with all chunks, I’ve deleted all index files for this tenant. Restarted loki. Stopped promtail on server with logs, remove positions for log files I want to digest again. Started promtail again.
expected result:
Logs are digested again and I can go through them.
Actual result:
New chunks with old logs are generated but index files references old chunks (nonexisting) again.
