How to get the difference between two time epochs in logs after correlating them

Grafana Dashboards
, , ,
1

Hi,

Below is a sample log that I am ingesting to Loki.

When an action starts:
“2025-03-13 12:48:01.413 TID=non-unique-id-like-1234 EXPID=unique-id-123 TASK=some-task-like-login STATUS=START EPOCH=unix-time-epoch”

When the same action ends:
“2025-03-13 12:48:01.413 TID=non-unique-id-like-1234 EXPID=unique-id-123 TASK=some-task-like-login STATUS=END EPOCH=unix-time-epoch”

I am trying to calculate the time difference between the END and START of the action by getting the END unix-time-epoch - START unix-time-epoch so I know how much time the task had taken to complete. I am planning to show this time variation over time for each of the tasks on a Grafana dashboard.

So far, following a below attached thread, I was able to use Pattern to extract labels and do extract fields and group by transformation to group the log entries by TID, EXPID, and TASK and currently it is visible as

Time tid expid task epoch (allValues) status (allValues)
2025-01-01 12:00:00 1 1111 A [“174…737”, “174…743”] [“START”, “END”]
2025-01-01 12:00:00 1 1111 B [“174…737”, “174…743”] [“END”, “START”]

What I wish to do is to

  1. Filter out rows that does not contain START and END for status, order does not matter.
  2. If valid row after filtering, subtract the higher epoch from lower epoch to get the time for that row.
  3. Convert the rows to a timeseries so I can plot this on grafana (To show time taken to task A to complete over time etc. )

How can I do this?

Found this closed issue. Provide Metric query to calculate time difference between logs · Issue #4659 · grafana/loki · GitHub related to this.
I found a close match to my question here Loki query to find difference between 2 timestamps - #2 by clevernessisamyth but still no could not figure out how to map that to my usecase.

Thanks in advance.

2

I don’t think you can do this in LogQL alone. I don’t know for sure if it would work, but you might be able to do what you are looking to do with Grafana:

  1. In grafana, create two queries, one for START one for END.
  2. Use transformations and join the two queries together by tid/expid/task.
  3. Use another transformations and subtract column END from column START.
1 Like