What Grafana version and what operating system are you using?
v10.3.4, Linux
What are you trying to achieve?
We are working on allowing users to access Grafana by port-forwarding from Kubernetes as admin.
How are you trying to achieve it?
I’m running Grafana helm chart version 7.3.9 (latest). I’m running Grafana through terraform and attempting to access by port-forwarding from local command line, eg: kubectl port-forward svc/grafana 3001:80 and logging into localhost:3001 locally.
Some things we’ve tried:
- I’ve tried running this on Grafana 10.3.4 and 10.2.2.
- I also tried to run this with Chart version 6.56.1 (we updated the chart to confirm the chart version wasn’t the issue here).
- I am able to access dashboards/alerts/metrics etc. by setting anonymous auth, but when I set anonymous auth, attempting to log in with username/password still doesn’t work.
What happened?
I’m getting a user/pass failed on attempting to log in.
What did you expect to happen?
I expect to be able to login as admin with admin/password credentials.
Can you copy/paste the configuration(s) that you are having problems with?
Helm chart:
image:
repository: grafana/grafana
tag: 10.3.4
adminPassword: password
datasources:
datasources.yaml:
apiVersion: 1
datasources: []
grafana.ini:
log:
level: debug
users:
auto_assign_org_role: Viewer
persistence:
enabled: true
serviceMonitor:
enabled: true
sidecar:
alerts:
enabled: true
logLevel: DEBUG
useStatefulSet: true
This is the grafana.ini from within the pod:
[analytics]
check_for_updates = true
[grafana_net]
url = https://grafana.net
[log]
level = debug
mode = console
[paths]
data = /var/lib/grafana/
logs = /var/log/grafana
plugins = /var/lib/grafana/plugins
provisioning = /etc/grafana/provisioning
[server]
domain = ''
[users]
auto_assign_org_role = Viewer
When I SSH into the relevant pod, I see the env variables GF_SECURITY_ADMIN_PASSWORD=password and GF_SECURITY_ADMIN_USER=admin, so admin/password should be the correct credentials here.
Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
I see the following logs in Grafana:
logger=authn.password t=2024-04-19T22:10:51.793268812Z level=debug msg="Failed to authenticate password identity" client=grafana error="[password-auth.invalid] invalid password"
logger=authn.service t=2024-04-19T22:10:51.799198887Z level=info msg="Failed to authenticate request" client=auth.client.basic error="[password-auth.failed] failed to authenticate identity: [password-auth.invalid] invalid password"
logger=context userId=0 orgId=0 uname= t=2024-04-19T22:10:51.799254052Z level=info msg= error="[password-auth.failed] failed to authenticate identity: [password-auth.invalid] invalid password" remote_addr=[::1] traceID=
logger=context userId=0 orgId=0 uname= t=2024-04-19T22:10:51.799295361Z level=info msg="Request Completed" method=POST path=/api/admin/provisioning/alerting/reload status=401 remote_addr=[::1] time_ms=18 duration=18.792781ms size=121 referer= handler=/api/admin/provisioning/alerting/reload
The Alerts sidecar yields a similar error:
{"time": "2024-04-19T22:10:51.778147+00:00", "taskName": null, "msg": "Writing /etc/grafana/provisioning/alerting/alerts.yaml (ascii)", "level": "INFO"}
{"time": "2024-04-19T22:10:51.799953+00:00", "taskName": null, "msg": "POST request sent to http://localhost:3000/api/admin/provisioning/alerting/reload. Response: 401 Unauthorized {\"extra\":null,\"message\":\"Invalid username or password\",\"messageId\":\"password-auth.failed\",\"statusCode\":401,\"traceID\":\"\"}\n", "level": "DEBUG"}
Did you follow any online instructions? If so, what is the URL?
I checked Grafana auth docs.
I must have missed that.