Grafana Regular expression to extract value from a log line

I want to extract a value from a log line. I’m using filebeat as data source. Using the extracted value, I want to create a graph panel.
Example:

1/22/2024#18:55:54:919 #API#RequestAndResponseLoggingFilter#NA#NA#NA#-#SG#default task-26982#PRD_SG_eBBSPRI##uetr:: null##18:55:54#O#-#-#-#APIServices#-#**60**#-##POST//rcb-api/rcb/api/v1/payment-transactions/ null

From the above log line, i want to extract 60. which is transaction posting time taken.

I’m currently using below query, but not getting the value as expected.

message:(#O#-#-#-#APIServices#-#(\d+)#-)

image

@sowdenraymond - Thanks for your reply. We’ve tried this website but this query is working only in the regex website and not in grafana. Grafana doesnt extract any data using this regex. when I was reading on this, found below response in one of the thread.

@yosiasz - Go, hence Grafana uses RE2 regexp which does not support a lot of things. Check it out here: Syntax · google/re2 Wiki · GitHub .
I had the same issue, tested on regex101, then the pattern did not work in Grafana, then learned that RE2 and PCRE2 are two different things.”

Help with any other solution.

1 Like

you have to use the golang flavor in the regex101 tester

1 Like

@joachimschiewek - Thanks for the reply. using ‘Golang’ gives the result. But not able to use that extracted value to form a Graph / TimeSeries board. Since I cant find an option to use the extracted value. If i select count, then the value is getting changed, instead of showing the actual extracted value, its being counted.

do we need to use Transformation? pls help with this.

Unfortunately, I have no experience with Elasticsearch as data source.

You are not using filebeat datasource, but you are using elasticsearch datasource.
An Elasticsearch datasource supports only Lucene query, it doesn’t support regex query. I don’t believe there is some Lucene function which will provide parsing functionality as you need, so I would recommend to configure the filebeat (or that process which ingest data into Elasticsearch) to parse and save that number directly in some field that you can then query with Lucene query from Grafana.