Springboot app attributes with otel collector

Grafana Configuration
1

anyone using otel collector to send springboot(2.x) app metrics into grafana.com
im using below in processor otel.conf
but metrics inside grafana are showing job value as “java/myapp” instead of “myapp” as per below
other attributes do not have this issue
only job value gets namespace/myapp as attribute for job attribute ?

  - action: insert
    key: job
    value: "myapp"
  - action: insert
    key: instance
    value: "myapp"
  - action: insert
    key: service.name
    value: "myapp"
  - action: insert
    key: service.namespace
    value: "java"
2

please provide reproducible example

3

im deploying app in aws beanstalk using otelcol as second container ,
I will try to retest using public image app using docker compose and then in beanstalk to compare if its only happening in aws beanstalk environment.

4

Please provide your otel.conf

5

tested removing namespace value and then job value is exactly whats defined without namespace/jobvalue like below

  - action: insert
    key: job
    value: "sbdemo-dev1"

you can test with any demo app using these in docker-compose
OTEL_RESOURCE_ATTRIBUTES=service.name=sbdemo-dev1,service.namespace=java1,job=sbdemo-dev1,application=sbdemo-dev1

urrl without namespace
https://devmonitor.grafana.net/a/grafana-app-observability-app/services/myapp

urrl with java namespace and doesnt showup in services but showed up after long time
https://devmonitor.grafana.net/a/grafana-app-observability-app/services/java---myapp

note: not allowing to upload yaml file
otel.conf

processors:
  batch:
    send_batch_size: 10  # Number of objects which will be sent in each batch
    #send_batch_size: 8192  # Number of objects which will be sent in each batch
    timeout: 10s  # Time after which a batch will be sent anyway, regardless of size
    send_batch_max_size: 0  # Upper limit of the batch size, 0=no upper limit
  attributes:
    actions:
      - action: insert
        key: env
        value: "dev"
      - action: insert
        key: application
        value: "sbdemo-dev1"
      - action: insert
        key: job
        value: "sbdemo-dev1"
      - action: insert
        key: instance
        value: "sbdemo-dev1"
      - action: insert
        key: service.name
        value: "sbdemo-dev1"
      #- action: insert
      #  key: service.namespace
      #  value: "java"
      - action: insert
        key: service.version
        value: "1.0.0"
      - action: insert
        key: deployment.environment
        value: "dev"
6

You are using attributes processor (that’s not obvious from your first post). So check the doc for that processor: opentelemetry-collector-contrib/processor/attributesprocessor at main · open-telemetry/opentelemetry-collector-contrib · GitHub

insert: Inserts a new attribute in input data where the key does not already exist.

“where the key does not already exist” - your app already provides that attributes, so OTEL collector won’t insert that key in that case

Read the doc (that’s your good friend) and I think upsert operation will be better fit for you (not insert)

7

thx @jangaraj

found below about job attribute

However, <service.namespace>/<service.name> or <service.name> (if namespace is empty), is added as job label and service.instance.id is added as instance label to every metric.

the default springboot dashboard for metrics is not picking up all metrics , will need some changes
posted another thread below