Cannot start grafana agent with apache integration

Hi all,

I have installed the grafana-agent as instructed by the installer on Grafana Cloud, and I was trying to follow the installation guide for the Apache integration. Unfortunately it does not work.

The part that collects logs is OK. But the part where the integration is setup is not working properly.

Here is my config:

integrations:
  agent:
    enabled: true
    relabel_configs:
    - action: replace
      source_labels:
      - agent_hostname
      target_label: instance
    - action: replace
      target_label: job
      replacement: "integrations/agent-check"
    metric_relabel_configs:
    - action: keep
      regex: (prometheus_target_.*|prometheus_sd_discovered_targets|agent_build.*|agent_wal_samples_appended_total|process_start_time_seconds)
      source_labels:
      - __name__
  apache_http:
    enabled: true
    instance: hostname
    scrape_uri: 'http://localhost/server-status?auto'
  agent:
    enabled: true
    metric_relabel_configs:
    # required for apache integration.
    # scraping agent endpoint is required for apache histogram metric collection.
      - source_labels: [exported_job]
        target_label: job
      - source_labels: [exported_instance]
        target_label: instance
      - regex: (exported_instance|exported_job)
        action: labeldrop
  prometheus_remote_write:
  - basic_auth:
      password: XXXXX
      username: XXXX
    url: https://XXXXX/api/prom/push
logs:
...

I am getting this error:

Jun  5 12:33:59 grafana-agent[969634]: 2023/06/05 12:33:59 error loading config file /etc/grafana-agent.yaml: yaml: unmarshal errors:
Jun  5 12:33:59 grafana-agent[969634]:   line 20: field agent already set in type integrations.ManagerConfig
Jun  5 12:33:59 systemd[1]: grafana-agent.service: Main process exited, code=exited, status=1/FAILURE

If I comment out the apache_http and the following agent sections, it works correctly.

Any idea?

The documentation suggests to add a DUPLICATE agent key in the Yaml file. The full example of a supposedly complete configuration does the same, see here: Apache HTTP server integration | Grafana Cloud documentation

This looks like invalid yaml to me.

The following works (I merged the contents of the two agent keys).

integrations:
  agent:
    enabled: true
    relabel_configs:
    - action: replace
      source_labels:
      - agent_hostname
      target_label: instance
    - action: replace
      target_label: job
      replacement: "integrations/agent-check"
    metric_relabel_configs:
    - action: keep
      regex: (prometheus_target_.*|prometheus_sd_discovered_targets|agent_build.*|agent_wal_samples_appended_total|process_start_time_seconds)
      source_labels:
      - __name__
    - source_labels: [exported_job]
      target_label: job
    - source_labels: [exported_instance]
      target_label: instance
    - regex: (exported_instance|exported_job)
      action: labeldrop
  apache_http:
    enabled: true
    instance: (...myhostname...)
    # job is integrations/apache_http by default
    scrape_uri: 'http://localhost/server-status?auto'
  prometheus_remote_write:
(... etc ...)