Grafana-agent (linux) won't run as grafana-agent after being run as root [SOLVED]

  • What Grafana version and what operating system are you using?
    grafana-agent 2024-01-19T11:55:16Z
  • What are you trying to achieve?
    Run as user grafana-agent from system.d
  • How are you trying to achieve it?
    in /lib/systemd/system/grafana-agent.service:


  • What happened?

For debugging purposes I executed the grafana-agent go script as root user. Following that, the systemd script now fails when setting the daemonisation user to the inbuilt ‘grafana-agent’ user. When the User= line is commented out and grafana-agent is run as root, it executes fine.

This is not optimal from a security perspective.

I assume that there may be some file permission somewhere that is being set which results in the script not executing, but I can’t find it.

  • What did you expect to happen?

grafana-agent runs as a daemon under the grafana-agent user.

  • Can you copy/paste the configuration(s) that you are having problems with?

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.

grafana-agent.service: Start request repeated too quickly.

  • Did you follow any online instructions? If so, what is the URL?

As always, found it just after asking for help.

The file /tmp/positions.yaml is created with the permissions of the running user and needs to be manually deleted, or a ‘permission denied’ error is generated.