Hi everyone,
We are planning to migrate our data and logs from elastic to loki. Before migration, we would like to know in detail if loki can ensure dynamic recalculation of data fields, just like happens in elasticsearch. Can we search for logs in loki based on different aggregations?
Also, can loki handle detokenization of JSON event records(to be searchable and aggreable)?
-
Loki does not have feature such as script field, so that’s a no for dynamic recalculation.
-
Not sure what you mean by searching for logs based on different aggregations.
-
Not quite sure what you mean by detokenization of JSON. Do you mean forming new JSON from other fields, or…?
In general Loki is lacking in terms flexibility on the query language. In exchange you get more efficiency and scalability, and possibly cost (we saved quite a bit of money switching from elasticsearch to Loki).
Hopefully others can provide you with better answers in terms of solutions to ease the migration pain. We were fortunate that we weren’t exactly power users of ES, and the functions that we couldn’t do in Loki we either transform Loki logs to Prometheus metrics and have been able to do some of those there, or scripts with API calls if we really have to.