Multi-Line Logging Not Working

Grafana Alloy
1

I’ve followed this for the Grafana Agent in flow mode:

And I have the following in my Helm values.yaml:

agent:
   configMap:
      create: true
      content: |
         ...etc.
        local.file_match "pods" {
        path_targets = discovery.relabel.pods.output
      }

      loki.source.file "pods" {
        targets = local.file_match.pods.targets
        forward_to = [loki.process.pods.receiver]
      }

      loki.process "pods" {
        stage.multiline {
          firstline     = "^\\d{4}-\\d{2}-\\d{2}"
          max_wait_time = "20s"
        }
      forward_to = [loki.write.endpoint.receiver]
      }     

      loki.write "endpoint" {
        endpoint {
            url = "https://loki.company.com/loki/api/v1/push"
        }
      }

I want to ensure logs like the following that are output to stdout are logged as a single multi-line log, rather than each line logged separately:

2023-10-03 07:01:43.925 [pool-11-thread-1]  WARN  current - [com.company.app.ReportAgent:109] Error regenerating report 'central_test_workflow'
java.sql.SQLSyntaxErrorException: Unknown column 'TypeViews.ordinal' in 'field list'
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeQuery(ClientPreparedStatement.java:1003)
	at org.apache.tomcat.dbcp.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
	at org.apache.tomcat.dbcp.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
	at uk.co.itg.integration.utils.database.RecordingPreparedStatementDelegate.executeQuery(RecordingPreparedStatementDelegate.java:23)
	at uk.co.itg.integration.utils.database.RecordingPreparedStatement.executeQuery(RecordingPreparedStatement.java:98)
	at com.company.app.Report.regenerate(Report.java:207)
	at com.company.app.Report.refresh(Report.java:116)
	at com.company.app.ReportAgent.tickle(ReportAgent.java:98)
	at com.company.app.ReportAgent$2.run(ReportAgent.java:125)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)

But this isn’t happening; the logs are still showing as single log line entries in Grafana with Loki set to the datasource in Explore mode.
What am I doing wrong?

1 Like
2

Hello.
Any solution found in the meantime ?
Looking at kind of the same issue right now.
I found that more often than not, using `` with regex helps (instead of " " ). But here I’m still trying to figure why the regex doesn’t pick.
(I can’t imagine we’re the only persons trying to achieve this.)

I’m working with the Grafana Agent Flow, but syntax looks exactly the same so I wouldnt be surprised what works for you would work for me (and the other way as well).

If you found a solution, please share.
If I find something, I’ll update this thread.

Shameful edit:
I just figured my regex and the block worked fine, I just messed up the previous “forward_to”, so the block was bypassed…

here is the block that worked for me:

loki.process "multiline_log" {
    forward_to = [loki.relabel.dynamic_labels.receiver]
    stage.multiline {
        firstline       = `^[0-9]{1,2}/[0-9]{1,2}/[0-9]{4}`
    }
}

Log line looks like:
6/25/2024 2:30:15 PM - INFO: Errors occurred during BlahBlahBlah Interesting log.

1 Like