Unable to add labels to the logs

Grafana Loki
,
1

I want to add labels to the logs, before sending them to loki, from promtail, based on the log text. Following is my promtail_config code

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://Loki_IP/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log
  
  pipeline_stages:
    - regex:
        expression: '.*Test.*'
        labels:
          contains_test: "test"

But, it is not able to add Labels to the logs containing the world Test. Any idea whats wrong with it? I have also tried following

pipeline_stages:
  - match:
      selector: '{job="varlogs"}'
      stages:
        - regex:
            expression: '.*Test.*'
        - labels:
            contains_test: "true"

Both of the approaches are not working

2

You need to use the match block to perform a conditional label add. See match | Grafana Loki documentation

3

Yes, I have tried that also. You can see my 2nd solution, but still its not working for me? I have tried this approach

pipeline_stages:
  - match:
      selector: '{job="varlogs"}'
      stages:
        - regex:
            expression: '.*Test.*'
        - labels:
            contains_test: "true"
4

That’s also incorrect. What you are doing is matching job = "varlogs". Instead you should do regex first, with a group capture, then match the group to a string, then set label there.

A line or two of example logs would also be helpful.

5

Do you have any working example of it? If yes, can you please share the code snippet?

6

If you can give me a couple of samples I can come up with something.

7

Sample means? Like console logs? I am doing POC, so I create system logs like with logger, i.e. logger "Testing123"

8

Here is my sample log:

this is a Test log
this is a normal log
this is a test log but lower case

Promtail configuration:

---
server:
  disable: true

positions:
  filename: /tmp/positions.yml

clients:
- url: http://localhost:3100/

scrape_configs:
  - job_name: test
    static_configs:
    - targets:
        - localhost
      labels:
        __path__: /tmp/test.log
        job: test
    pipeline_stages:
      - match:
          selector: '{job="test"} |= "Test"'
          stages:
            - static_labels:
                contains_test: "true"

Dry run results:

# promtail -config.file=/tmp/promtail.conf --dry-run --inspect
Clients configured:
----------------------
url: http://localhost:3100/
batchwait: 1s
batchsize: 1048576
follow_redirects: true
enable_http2: true
backoff_config:
  min_period: 500ms
  max_period: 5m0s
  max_retries: 10
timeout: 10s
tenant_id: ""
drop_rate_limited_batches: false
stream_lag_labels: ""

level=info ts=2023-10-23T18:59:15.319808277Z caller=promtail.go:133 msg="Reloading configuration file" md5sum=92d67002d7c42a4b6647ee2841d00198
level=info ts=2023-10-23T18:59:15.32223744Z caller=main.go:174 msg="Starting Promtail" version="(version=2.9.2, branch=HEAD, revision=a17308db6)"
level=warn ts=2023-10-23T18:59:15.322358666Z caller=promtail.go:260 msg="disable watchConfig" reason="promtailServer cast fail"
[inspect: static_labels stage]: 
{stages.Entry}.Entry.Labels:
	-: {filename="/tmp/test.log", job="test"}
	+: {contains_test="true", filename="/tmp/test.log", job="test"}
2023-10-23T18:59:20.323526786+0000	{contains_test="true", filename="/tmp/test.log", job="test"}	this is a Test log
2023-10-23T18:59:20.324894507+0000	{filename="/tmp/test.log", job="test"}	this is a normal log
2023-10-23T18:59:20.324897111+0000	{filename="/tmp/test.log", job="test"}	this is a test log but lower case
level=info ts=2023-10-23T18:59:20.323145568Z caller=filetargetmanager.go:361 msg="Adding target" key="/tmp/test.log:{job=\"test\"}"
level=info ts=2023-10-23T18:59:20.323269625Z caller=filetarget.go:313 msg="watching new directory" directory=/tmp
ts=2023-10-23T18:59:20.323493051Z caller=log.go:168 level=info msg="Seeked /tmp/test.log - &{Offset:0 Whence:0}"
level=info ts=2023-10-23T18:59:20.323530403Z caller=tailer.go:145 component=tailer msg="tail routine: started" path=/tmp/test.log
2 Likes
9

This worked for me. Thanks @tonyswumac

10

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.