Promtail: "open /var/log/boot.log: permission denied"

Grafana Loki
1

I’m having an issue with some sort of permissions. In Grafana and looking at the logs, I’m always getting an error on the log under “systemd-journal” and it states: “open /var/log/boot.log: permission denied” for several logs that are owned by root. Now I’ve read places that these logs are owned by root and give many people headaches, but specifically to Promtail, I’m not sure what I should do? Do I add Promtail to a specific group or something? Or maybe I’m missing something else?

2

Did you try running promtail as root? for me it fixed it.

3

Hmm, excuse my ignorance, but how would I do that, or tell if I’m doing it already? Not sure if it matters, but whenever I install new programs and such, I don’t use the root account (because of bad practices), so my main account has the “sudo” privileges. And I do it that way.

EDIT: Actually I got it to work. Had to manually go into the folder as root, and fire to start the program, and just like @dror1212 stated the errors went away. However, my question is now, is how can I start promtail as root as a service so it starts with root when the computer boots up?

EDIT 2: Well, a little more research and I found my solution: I edited the .service under
/etc/systemd/system for promtail, and changed the user from the promtail user to root, and it worked like a charm. Solved my own issue. I’m sure this info will come in handy for others if they run into the same problem.

1 Like
4

I don’t think it is good practice to run promtail as root. It is better to have a promtail usr and add that user to the systemd-journal group:

$ sudo adduser --system promtail
$ cd /var
$ var/ sudo setfacl -R -m u:promtail:rX log
$ sudo chown promtail:promtail /tmp/positions.yaml
$ sudo usermod -a -G systemd-journal promtail
3 Likes
5

I have to agree with @hubba here, running any application that doesn’t need true root access should not be done. There are better ways to do it via permissions, group associations, and ACLs.

For those on SUSE, this was my fix (assuming you installed Loki and Promtail via Zypper) with the help of hubba:

sudo zypper in acl
sudo useradd -c "Promtail System User" -d "/etc/loki" -M -r promtail -s "/sbin/nologin"
cd /var
sudo setfacl -R -m u:promtail:rX log
sudo usermod -a -G systemd-journal promtail

Then I just needed to make sure the SystemD service file was updated to run as promtail instead of loki.

1 Like
6

There are some additional details on hardening of the promtail service here: Getting Logs Into Loki With Promtail

7

I setup promtail following those guides successfully about 3-4 months ago, but as of an update the last 3 weeks it no longer works, I am only able to tail /var/log/* and other static_config paths when setting promtail to run as root.

8

I was fighting with a similar issue, primarly shipping the journal to loki. What resolved the issue for me is to give the user r-x permissions on /var/log and all the subfolders with acls.

9

I had the same problem, I wanted to tail logs from four “.log” files in a folder inside /var/log.
Giving 755 permission to the folder that contains the log files, saved my day.

10

So this only works until new files are created or files are rotated (for example for apache2 after a day. So this works until the end of the day).
So giving /var/log the 755 is the only way to solve this?

11

Try adding a “d” (default) at the beginning of the ACL:
setfacl -R -m d:u:promtail:rX log
This will work for future file creations.

To apply to the existing files/directories, just remove the “d”. In my installation, I used both:
First:
setfacl -R -m u:promtail:rX log
Then:
setfacl -R -m d:u:promtail:rX log