structuredMetadata vs label escaping

Grafana Loki
, ,
1

I’m using Grafana Loki for storing logs. I am using structuredMetadata for faster lookup. I’m ingesting and reading data to/from Loki via http API. When I want to query by some structuredMetadata that contains escaped characters, I have to use a different expression than the one I use in label query expression, although I’d expect that both expressions should be the same.

For example, I ingest one record with one label label: "value and one structuredMetadata struct_metadata: "value (both contain double quote) using this request (JSON payload):
POST <loki-gateway-host>/loki/api/v1/push

{
  "streams" : [ {
    "stream" : {
      "label" : "\"value"
    },
    "values" : [ [ "<unixEpochNanoseconds>", "<logLine>", {
      "struct_metadata" : "\"value",
    } ] ]
  } ]
}

When I want to query all events with this label and with this struct_metadata, I send request GET <loki-gateway-host>/loki/api/v1/query_range with this query param:

{label="\"value"} | struct_metadata="\"value"

The stream selector using event_source="\"label-value" correctly filters records with label value "label-value. However, the structMetadata filter expression does not work. I have to change the query to

{label="\"value"} | struct_metadata="\\\"value"

In other words, I have to add extra backslash.

Is that expected behaviour? Why the label value can be compared without extra escaping / why the structMetadata value must contain extra escaping?

Thank you for your response!

2

Try wrapping the label value filter into single quotes or into backticks instead of double quotes:

{label='"value'} | struct_metadata='"value'
3

Hi! Thank you for the suggestion. Singe quotes are invalid (invalid syntax), only double quotes or backticks can be used. I have tried backticks but it behaves the same as double quotes.

I have noticed that Loki treats label values escaping differently from struct_metadata values, it may not be a problem of incorrect query. For example, if you push this:

{
  "streams" : [ {
    "stream" : {
      "label" : "\"value"
    },
    "values" : [ [ "1749031169275416000", "whatever", {
      "struct_metadata" : "\"value"
    } ] ]
  } ]
}

and then search by query

{label="\"value"} | struct_metadata="\"value"

you get in response

{
    "status": "success",
    "data": {
        "resultType": "streams",
        "result": [
            {
                "stream": {
                    "detected_level": "unknown",
                    "label": "\"value",
                    "service_name": "unknown_service",
                    "struct_metadata": "\\\"value"
                },
                "values": [
                    [
                        "1749031169275416000",
                        "whatever"
                    ]
                ]
            }
        ],
        "stats": {

Correct me if I’m wrong but I think that this is a bug. In my opinion Loki incorrectly stores struct_metadata value as "\\\"value" although it should be stored as “"value” (same as label value).