Optional character in Loki Pattern Parser

I’m trying to extract GET query parameters from URIs in an nginx log using the Pattern Parser

Given a typical nginx access log line like this

some stuff I want /website.com/route/path?query="sparsely happens" some more stuff I want to match

I can easily extract the full url with the query using

pattern `<_> - <_> <_> "<method> <url> <_>" <status> <_> <_> "<_>" <_>`

but if I want to split them and use the following

pattern `<_> - <_> <_> "<method> <url>?<query> <_>" <status> <_> <_> "<_>" <_>`

and the log line happened not to have a query (aka just a path like “website.com/route/path”) then url becomes everything from the method onwards (which makes sense). So how do I define the ?<query> as optional?

The pattern parser doesn’t support optional fields. Perhaps you can use multiple parsers in a row here. First run the pattern parser with the full url. Then add a regexp that can extract the url and query parts as labels. A line format expression can then be used to print the new labels.

Hi @aminnaggar ,

I realize this is not a direct answer to your question but as I have dealt a bit with Nginx logs lately I thought I would chime in anyway.

I have been testing Loki over the past few months and I have come to the conclusion (big surprise) that logs will work best if they are properly structured. Yes, Nginx logs do have a well known structure, in a way but they are not structured as in broken down into fields as JSON or logfmt logs are.

This is why I decided to experiment with Nginx log formats. In my head you were talking about Kubernetes Nginx Ingress controller logs but that is not necessarily the case. Be it as it may, Nginx logs are pretty easy to structure as logfmt or JSON which will make them much more Loki friendly. If that is an option for you… Happy to share some examples.