GitLab EE conflicts with Grafana Agent Prometheus port 9090

Environment:

  • Grafana Cloud stack
  • GitLab EE 16.6 on Debian 11
  • Hosted on a Google Cloud VM (Debian 11)

Issue Description:
I am experiencing a port conflict when integrating the Grafana Agent with GitLab EE, as outlined in the Grafana integration guide for GitLab. Specifically, both the built-in GitLab Prometheus exporter and the Grafana Agent are attempting to use port 9090. This conflict prevents the Grafana Agent from starting, as it fails when port 9090 is occupied by another service.

Attempted Solution:
To address this, I modified the /etc/gitlab/gitlab.rb file, changing the Prometheus listen address:

prometheus['listen_address'] = ':9099'

Queries:

  1. Is altering the Prometheus listen address to port 9099 in GitLab a viable long-term solution for this port conflict?
  2. Would it be more advisable to disable the Prometheus export on port 9090 within the Grafana Agent instead?

I’m seeking guidance on the best practices to resolve this conflict and ensure smooth integration of the Grafana Agent with GitLab EE.