Hi. I am working with NGINX logs. I am loging using json format. I have noticed NGINX creates a nice json entry per request with all the defined fields, which is nice, but in case of a field not having a value, it still sets it to an empty string, and it ends up getting to Loki database at the end.
What I would like to do is to have a easy way to remove this empty fields, to make the logs lighter. There is no need to keep these fields around. I find Promtail hard to work with Json editing and could not come up with a proper solution. The Json parser works fine, but when it comes to addition/removal/edition of fields, it quickly gets quite messi.
Thanks in advance.
First I just want to say that I hope you are not converting everything into labels.
Second, a bit of clarification, are you looking to alter the log message to remove empty fields, or are you looking to NOT make a field into labels if it’s empty?
Unfortunately neither is possible with Promtail (at least I can’t think of a way to do so dynamically). You will likely need log agent that allows you write inline script such as logstash to do dynamic filtering.
And lastly, my personal opinion is that you are probably over-engineering it a bit.
Hi Tony, thanks for the quick reply!
We have only 5 native labels used across all our servers, and that is enough. I tried to put everything (including levels) as json fields.
I am trying in this particular case to remove empty fields in the Promtail agent before pushing them to the Loki server. NGINX is generating, depending of the hit, several empty fields.
I have been really struggling in many cases to mutate, add or remove some of these fields. I find Promtail really not suitable for it, at least for now.
I just wanted to be sure I was not missing something obvious here.
I does not really hurt to send those entries without values, but if there was an easy way to prevent it, I’d do it. It is waste of resources.
Anyways, thanks for the official clarification.
That makes sense. Then I would recommend you to look into doing that in something like logstash. But I’ll say even in logstash it’s not going to be straight forward. You would have to probably write an inline ruby script to break apart the json, loop through all fields, discard the empty ones, and then combine them again into a new log string.
Thanks. Yes, Logstash could be solve this. But I am trying to migrate the company out Datadog, into this new environment, and just right now is not the right time to involve another big player as Logstash into the stack, (just for this).
So let’s have some empty fields for now 
Anyways, I’ll have this in mind for the future.
Thanks!
