How to add custom label with dynamic value in k6 logs with log-output forwarding?

Grafana k6 Extensions
1

The default message format for k6 with console.log is

time="2023-07-19T10:40:20Z" level=info msg="hello" source=console

I want to modify this message and let’s say add a custom label like trace=1234 method=login

I want to do this because the logs will be sent to Loki and then I can search by labels like method

How can I achieve this?

I know, I can achieve this by creating a custom javascript extension with a custom log method. I have tried this like following for testing purposes:

    package tracelogger
    
    import (
        "fmt"
    	"time"
    
        "go.k6.io/k6/js/modules"
    )
    
    // init is called by the Go runtime at application startup.
    func init() {
        modules.Register("k6/x/tracelogger", new(TraceLogger))
    }
    
    type TraceLogger struct{
    }
    
    func (l *TraceLogger) Log() {
    		currentTime := time.Now()
    		formattedTime := currentTime.Format("2006-01-02T15:04:05Z07:00")
    		fmt.Printf("time=\"%s\" level=info trace=1234 method=abc msg=\"%s\" source=console", formattedTime,  "hello")
    }

Now in the JS test file

    import tracelogger from 'k6/x/tracelogger';
    
    export const options = {};
    
    export function setup() { }
    
    export default function LoadTest() {
      tracelogger.log();
    }

It does print the output but in the standard console as expected

    time="2023-07-19T12:08:18Z" level=info trace=1234 method=abc msg="hello" source=console

Now the problem is I am also forwarding the log output to Loki

    k6 run --log-output=loki=http://localhost:3100/loki/api/v1/push,label.qa=k6 test.js

The log does not get forwarded to Loki, how do I log so that the log gets forwarded to loki?

I also tried with --log-format=raw and direct console.log

    console.log(`trace=1234 msg="aaaa"`);

but the labelling doesn’t work correctly in loki

enter image description here

5

Hi @noman637, Welcome to the forum!

First, AFAIK having a label with constantly changing values is not a good idea. There is a nuance to that, so you are likely fine for some of those.

On the k6 part:
What you are doing is not actually logging through the logger, so consequently it doesn’t go to loki.

What you are doing with fmt.Print is just writing directly to stdout the text you provide.

In order to get access to the logger you will need to use the (advanced) modules API

There isn’t a particular small example with just using the logger, but here is a line from another extension.

In order to add a label you will need to add it WithField.

but the labelling doesn’t work correctly in loki

This is are not labels that you have added this is just text, that is format how loki will format them if they were labels.

I think loki can extract labels from just text, but I can’t find concrete example and haven’t done this myself.

Hope this helps you!

1 Like
7

To achieve custom logging with labels in Loki, you can use the k6 console.log() function with a modified log format. However, for Loki to interpret the logs correctly, you need to use JSON log formatting and provide the appropriate labels.

Here’s an example of how to achieve this:

  1. In your custom Go extension (tracelogger.go), modify the Log function to output logs in JSON format:

goCopy code

package tracelogger

import (
	"encoding/json"
	"fmt"
	"time"

	"go.k6.io/k6/js/modules"
)

// init is called by the Go runtime at application startup.
func init() {
	modules.Register("k6/x/tracelogger", new(TraceLogger))
}

type LogMessage struct {
	Time   string `json:"time"`
	Level  string `json:"level"`
	Trace  string `json:"trace"`
	Method string `json:"method"`
	Msg    string `json:"msg"`
	Source string `json:"source"`
}

type TraceLogger struct{}

func (l *TraceLogger) Log() {
	currentTime := time.Now()
	formattedTime := currentTime.Format("2006-01-02T15:04:05Z07:00")
	msg := LogMessage{
		Time:   formattedTime,
		Level:  "info",
		Trace:  "1234",
		Method: "abc",
		Msg:    "hello",
		Source: "console",
	}

	logJSON, _ := json.Marshal(msg)
	fmt.Println(string(logJSON))
}
```[ciclo di sustanon 350](https://fitexpt.com/ciclo-di-sustanon-350/)
2 Likes
8

Thanks for the reply @wendllandtyrell40 :bowing_man:

To achieve custom logging with labels in Loki, you can use the k6 console.log() function with a modified log format. However, for Loki to interpret the logs correctly, you need to use JSON log formatting and provide the appropriate labels.

This seems to be what I meant by

I think loki can extract labels from just text, but I can’t find concrete example and haven’t done this myself.

I am still not certain I understand how it works.

And more importantly as I explained in the earleir parts of the post while fmt.Println outputs locally it doesn’t go through the logger adn consequntly k6 will not send it to loki.

And you can do that with console.log in js code directly:

let log = {
	time: "2006-01-02T15:04:05Z07:00",
	level: "info",
	trace: "1234",
	method: "abc",
	msg: "hello",
	source: "console",
};

console.log(JSON.stringify(log));

Unfortunately this likely won’t help you as that is again the message.

Basically one of them does not go through the logger so it doesn’t go to loki (using fmt.Println)

The other goes through the logger, but it can only become a message (console.log). This might work - again not that familiar with how much loki can do.

And the final option iwth using the actual logger and adding what logrus calls a field (WithField) which will add a label to loki, seems to be the most straightforward.

1 Like