How can we set up Fail2ban to protect our dashboard?

Grafana
21

Tried your syntax, however didn’t get very far. I also tried with/without the datepattern syntax.
Without the datepattern syntax I only get the below upon a failed login in the fail2ban logs.
It seems to be grabbing the first line and ignoring the second entry which contains the “Bad Request” entry even though it is in the regex filter.

2023-09-28 13:38:52,478 fail2ban.filter [1]: WARNING [grafana] Please check a jail for a timing issue. Line with odd timestamp: logger=authn.service t=2023-09-28T14:38:52.46676519-06:00 level=warn msg=“Failed to authenticate request” client=auth.client.form error=“[password-auth.failed] failed to authenticate identity: [identity.not-found] no user fund: user not found”

22

You definitely need the datepattern… fail2ban doesn’t know what to do with more than 3 digits of milliseconds, which is why you’re getting the “Line with odd timestamp” error. I think it also struggles when the date isn’t the first thing in the line, which is why my datepattern starts with ^(?:[^=]+=[^ ]* )+t=

As for which line fail2ban “sees,” it’s really just checking every line against the failregex and seeing if it matches.

Looking a little closer, the value in the error= doesn’t work with this regex. Can you try this instead?

msg=“Bad request” (?:[^=]+=(?:[^ ]|“[^”]") )+remote_addr=

edit: see below for a better regex

23

That worked, it logged the IP on the failed login attempt! Appreciate the help in getting the syntax correct.

24

Glad to help! I did edit the regex slightly after I posted initially… I think the edit is more correct, so you might want to update yours!

25

Think there might a typo in your edit.

msg=“msg=” vs msg="Bad…

26

Yup, good catch! I also changed out the + for * in the inner group to account for a key with an empty value.

27

I updated my Grafana to 10.1.2 and confirmed that the log messages for failed logins have changed.

After spending a little more time on this, I think the regex I provided above could result in false positives if some other event triggers a log with msg="Bad request". With that in mind, I’ve updated to what I believe will be a more reliable regex that focuses on the errror= section of the log entry. I also tweaked the datepattern to hopefully be more robust if the log format changes again.

[Init]
datepattern = ^(?:[^=]+=(?:[^ ]*|"[^"]*") )+t=%%Y-%%m-%%dT%%H:%%M:%%S.%%f(?:\d{0,6})%%z

[Definition]
failregex = error="\[password-auth\.failed\][^"]*" (?:[^=]+=(?:[^ ]*|"[^"]*") )*remote_addr=<ADDR>
29

Dear all, I have a dummy question, my grafan is installed in a docker on a synology NAS. How to add fail2ban ? An other specific container ?

30

I actually missed this update. Was having cloudflare IP’s banned as false positives. Am trying this newer regex.

31

@tkohhh Appears I got tagged with another false positive.
Here is the log snippet the IP is getting caught in resulting in a fail2ban for the IP.

logger=context userId=0 orgId=0 uname= t=2024-02-01T09:48:59.023048217-07:00 level=info msg="Request Completed" method=POST path=/api/ds/query status=401 remote_addr=172.*.*.* time_ms=1 duration=1.386157ms size=105 referer="https://*.*.net/*/kwtyVaAmz/**" handler=/api/ds/query

Would adding msg=“Request Completed” in the ignoreregex help in this case?

32

Are you sure you’re using the failregex I posted in this post? I ask because that failregex will only pick up log messages that have error="[password-auth.failed] in the log line. Your example doesn’t even have an error section, so I don’t see how my failregex could be triggered by that log line.

Is it possible you have another jail looking at the same log file that uses a different failregex?

33

It appears I did have a jail/filter for traefik access.log which may have spurred the ban. I have removed the traefik.conf filter to see if it will resolve the issue.