No result query when using timestamp stage in Promtail

[tuananh170489]

Hi there,

I’m trying to parse my app logs using Promtail and send them to Loki.

Here is an example of log lines

{"@timestamp":"2023-11-21T08:50:45.819Z","@metadata":{"beat":"filebeat","type":"_doc","version":"8.11.0"},"host":{"ip":["10.210.6.72"],"hostname":"fpro-price-streaming-api"},"message":"2023-11-21 15:50:44.083 [http-nio-80-exec-8] DEBUG o.s.w.f.CommonsRequestLoggingFilter - Before request [GET /stocks/snapshot?code=DP3, client=X.X.X.X]"}
{"@timestamp":"2023-11-21T08:50:37.818Z","@metadata":{"beat":"filebeat","type":"_doc","version":"8.11.0"},"message":"2023-11-21 15:50:36.825 [http-nio-80-exec-6] DEBUG o.s.w.f.CommonsRequestLoggingFilter - After request [GET /stocks/snapshot?code=DP3, client=X.X.X.X]","host":{"hostname":"myhostname","ip":["X.X.X.X"]}}

and here is my Promtail configuration

scrape_configs:
  - job_name: test-json
    static_configs:
      - labels:
          job: test-json
          __path__: /tmp/test.log
    pipeline_stages:
      - match:
          selector: '{job="test-json"}'
          action: keep
          stages:
            - json:
                expressions:
                  message: '"message"'
                  metadata: '"@metadata"'
                  timestamp: '"@timestamp"'
                  host: '"host"'
            - regex:
                expression: '^(?P<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}) \[(?P<thread>[^\]]+)\] (?P<loglevel>[A-Z]+) (?P<class>[^\s]+) - (?P<msg>.+)$'
                source: message
            - timestamp:
                source: time
                format: "2006-01-02 15:04:05.999"
            - output:
                source: message

In inspect mode, I saw the timestamp stage look like successful

But when I’m using logcli to query it, I get no result from query

Any ideas where to look at?

[tonyswumac]

I don’t see anything immediately wrong. I’d say first you should try to identify where the problem is. First I’d make sure the logs are actually sent to Loki and that your Loki instance is working properly. You can do so multiple ways, for example try to send logs via API call, check whether chunks are written to storage, try to read logs without any filter with API call, etc.

[tuananh170489]

I think there is a problem in the timestamp stage
Because when I remove the timestamp stage from my configuration file, it works properly