K6 integration with Prometheus

Grafana k6 Extensions
1

Hi K6 team,

I used Combine k6 OSS and Prometheus for better observability this blog to try and integrate K6 with Prometheus. Currently we are just running a proof of concept to get a better understanding of visualisation graphs. Our problem is that we don’t use Influx db in Grafana, but we do have Prometheus in Grafana. So this seemed like an awesome solution.

This is all locally for now, once this works we want to do the same but then in Jenkins.

I want to however use a Dockerfile for step 2

xk6 build --with github.com/grafana/xk6-output-prometheus-remote@latest

So I created this file based on the K6 Github Dockerfile.

FROM golang:1.17-alpine as builder
WORKDIR $GOPATH/src/go.k6.io/k6
ADD . .
RUN apk --no-cache add git
RUN CGO_ENABLED=0 go install -a -trimpath -ldflags "-s -w -X go.k6.io/k6/lib/consts.VersionDetails=$(date -u +"%FT%T%z")/$(git describe --always --long --dirty)"
RUN go install -trimpath go.k6.io/xk6/cmd/xk6@latest
RUN xk6 build --with github.com/grafana/xk6-output-prometheus-remote@latest
RUN cp k6 $GOPATH/bin/k6

COPY ./tests ./tests

FROM alpine:3.13
RUN apk add --no-cache ca-certificates && \
    adduser -D -u 12345 -g 12345 k6
COPY --from=builder /go/bin/k6 /usr/bin/k6

USER 12345
ENTRYPOINT ["k6"]
CMD [""]

When I then run

K6_PROMETHEUS_REMOTE_URL=http://localhost:9090/api/v1/write docker run --rm -i -v "$PWD:/app" k6-img-prom run -o output-prometheus-remote /app/tests/script.js

It gives me this error

invalid output type 'output-prometheus-remote', available types are: cloud, csv, influxdb, json, statsd"

Can you tell me what I am doing wrong and hopefully guide me in the right direction?
Thanks in advance!

Rody

3

Hi @rodyb

Welcome to the community forums! :wave:

I’ve checked your Dockerfile, and it seems to work for me (I excluded the fifth line only).

From the error message, I can guess that there were runs not the built version of the k6 (or the version constructed without xk6-output-prometheus). So I’d recommend rebuilding/retagging the image.

Below you can find a simple docker-compose.yml:

version: '3.7'

volumes:
  prometheus-data:

services:
  k6-prom:
    build: .
    volumes:
      - .:/app:delegated
    environment:
      - K6_PROMETHEUS_REMOTE_URL=http://prometheus:9090/api/v1/write
    command: run -o output-prometheus-remote /app/tests/script.js

  prometheus:
    image: prom/prometheus:latest
    ports:
      - 9000:9090
    volumes:
      - ./configs/prometheus:/etc/prometheus
      - prometheus-data:/prometheus
    command: --enable-feature=remote-write-receiver --config.file=/etc/prometheus/prometheus.yml

And the Dockerfile:

FROM golang:1.17-alpine as builder
WORKDIR $GOPATH/src/go.k6.io/k6
ADD . .
RUN apk --no-cache add git
# RUN CGO_ENABLED=0 go install -a -trimpath -ldflags "-s -w -X go.k6.io/k6/lib/consts.VersionDetails=$(date -u +"%FT%T%z")/$(git describe --always --long --dirty)"
RUN go install -trimpath go.k6.io/xk6/cmd/xk6@latest
RUN xk6 build --with github.com/grafana/xk6-output-prometheus-remote@latest
RUN cp k6 $GOPATH/bin/k6

# COPY ./tests ./tests

FROM alpine:3.13
RUN apk add --no-cache ca-certificates && \
    adduser -D -u 12345 -g 12345 k6
COPY --from=builder /go/bin/k6 /usr/bin/k6

USER 12345
ENTRYPOINT ["k6"]
CMD [""]

An advanced example also can be found in the extension repository.

let me know if that helps,
Cheers!

4

hi @olegbespalov,

Thanks for your quick reply it worked the initial error is gone, but I get another error now.

I used the prometheus.yml from the example folder in the repository

global:
  scrape_interval:     10s
  evaluation_interval: 10s

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ['localhost:9090']

remote_write:
  - url: "http://promscale:9201/write"

remote_read:
  - url: "http://promscale:9201/read"
    read_recent: true

Error

k6-prom_1 exited with code 0
prometheus_1  | ts=2022-01-05T11:06:33.351Z caller=dedupe.go:112 component=remote level=warn remote_name=55eff5 url=http://promscale:9201/write msg="Failed to send batch, retrying" err="Post \"http://promscale:9201/write\": dial tcp: lookup promscale on 127.0.0.11:53: no such host"
prometheus_1  | ts=2022-01-05T11:07:36.070Z caller=dedupe.go:112 component=remote level=warn remote_name=55eff5 url=http://promscale:9201/write msg="Failed to send batch, retrying" err="Post \"http://promscale:9201/write\": dial tcp: lookup promscale on 127.0.0.11:53: no such host"
prometheus_1  | ts=2022-01-05T11:08:36.214Z caller=dedupe.go:112 component=remote level=warn remote_name=55eff5 url=http://promscale:9201/write msg="Failed to send batch, retrying" err="Post \"http://promscale:9201/write\": dial tcp: lookup promscale on 127.0.0.11:53: no such host"
prometheus_1  | ts=2022-01-05T11:09:39.605Z caller=dedupe.go:112 component=remote level=warn remote_name=55eff5 url=http://promscale:9201/write msg="Failed to send batch, retrying" err="Post \"http://promscale:9201/write\": dial tcp: lookup promscale on 127.0.0.11:53: no such host"
5

It’s because the example from the repository is using a Promscale.

Here is a minimal example of the configs/prometheus/prometheus.yml that I used in the docker-compose of my first message.

global:
  scrape_interval: 30s
  scrape_timeout: 10s

scrape_configs:
  - job_name: services
    metrics_path: /metrics
    static_configs:
      - targets:
          - 'prometheus:9090'

Hope that helps :slight_smile:

6

Hi @olegbespalov,

Thanks for the quick help ! This really made my day! :slight_smile:

1 Like