Hello,
We are using FluentD CSV parser to ship logs to Loki. Logs have simple structure: Starting Time, Finish Time of the event with unique Task ID and Message field.
Example:
Task ID:1 , Starting Time: DD:MM:YY Time Finished Time: DD:MM:YY Time and Message: " Send " or " Received " depending on the event.
Goal is to take first log line, search for " Send ", extract Starting Time, and then query the next line with Task ID greater than previous started event and extract the finished time. The message in the later has to contain " Received "
Finally just subtracting the finished from the starting to calculate latency.
However, we are having hard time to use Loki Ql for this use case. Since it requires querying the two different lines at same time and extracting values based on string message present in these lines.