hibaos
July 24, 2023, 6:01pm
1
Hi,
I’m deploying Loki with Grafana Community Kubernetes Helm Charts | helm-charts and using the following configuration for the S3 auth:
...
loki:
storage:
type: s3
s3:
accessKeyId: <grafana-loki-s3-acceskeyid>
secretAccessKey: <grafana-loki-s3-secretaccesskey>
...
My question is:
How can I read the values for accessKeyId and secretAccessKey from a k8s secret?
Supposing I have the following secret, how do I use it in the above example?:
apiVersion: v1
kind: Secret
metadata:
name: loki-secrets
namespace: grafana-loki
data:
grafana-loki-s3-accessKeyId: **************
grafana-loki-s3-secretAccessKey: **************
type: Opaque
TIA!
1 Like
b0b
July 27, 2023, 7:15am
2
1 Like
hibaos
July 28, 2023, 12:35pm
3
Thanks @b0b !
I have now something like this in my Helm values file for deploying Loki and it works nicely
loki:
...
storage:
...
s3:
endpoint: "${GRAFANA-LOKI-S3-ENDPOINT}"
accessKeyId: "${GRAFANA-LOKI-S3-ACCESKEYID}"
secretAccessKey: "${GRAFANA-LOKI-S3-SECRETACCESSKEY}"
...
backend:
...
extraArgs:
- '-config.expand-env=true'
extraEnv:
- name: GRAFANA-LOKI-S3-ENDPOINT
valueFrom:
secretKeyRef:
name: loki-secrets
key: grafana-loki-s3-endpoint
- name: GRAFANA-LOKI-S3-ACCESKEYID
valueFrom:
secretKeyRef:
name: loki-secrets
key: grafana-loki-s3-accessKeyId
- name: GRAFANA-LOKI-S3-SECRETACCESSKEY
valueFrom:
secretKeyRef:
name: loki-secrets
key: grafana-loki-s3-secretAccessKey
...
1 Like
Can you add an example of your secret.yaml
(without the secrets of course)? did you added it manually to the cluster or you’ve put the YAML inside templates
folder under the helm chart?
hibaos
August 3, 2023, 6:58am
5
Hi @lekspek
Sure!
I use Argo CD to deploy Loki using its Helm chart and the configuration mapping the secrets as environment variables is defined in the values section for the helm chart. The secret file is located in another directory, that argocd reads when deploying/updating Loki.
The secret file looks something like:
apiVersion: v1
kind: Secret
metadata:
name: <loki-secrets-name>
namespace: <loki-namespace>
type: Opaque
data:
grafana-loki-s3-accessKeyId: <base64-hashed string>
grafana-loki-s3-secretAccessKey: <base64-hashed string>
grafana-loki-s3-endpoint: <base64-hashed string>
So basically you didn’t use Pre-defined variables like described here: Grafana Loki configuration parameters | Grafana Loki documentation
You’ve just created a new environment variable and pointed the global s3 configuration to it, right?
hibaos
August 3, 2023, 1:30pm
7
As I use the Loki Helm chart I also use its values.yaml file.
storage:
...
s3:
endpoint: "${GRAFANA-LOKI-S3-ENDPOINT}"
accessKeyId: "${GRAFANA-LOKI-S3-ACCESKEYID}"
secretAccessKey: "${GRAFANA-LOKI-S3-SECRETACCESSKEY}"
...
Where ‘endpoint’, ‘accessKeyId’ and ‘secretAccessKey’ are variables used by the helm chart. When processed results in the end in something like:
storage_config:
endpoint: ${endpoint}
access_key_id: ${accessKeyId}
secret_access_key: ${secretAccessKey}
...
Yes, it can be quite confusing
Hi all,
I have followed these steps and I am using sealed secrets to deploy the secret before deploying loki.
The sealed secrets controller creates the secret but then the grafana-agent-operator takes ownership of the secret and wipes the data values. Is there a setting somewhere to stop the grafana-agent-operator taking ownership of the secret.
apiVersion: v1
kind: Secret
metadata:
creationTimestamp: "2024-03-01T09:30:22Z"
labels:
app.kubernetes.io/managed-by: grafana-agent-operator
name: loki-secrets
namespace: logging
ownerReferences:
- apiVersion: bitnami.com/v1alpha1
controller: true
kind: SealedSecret
name: loki-secrets
uid: 416ec23a-d50b-4470-aed6-fa5870f6a087
- apiVersion: monitoring.grafana.com/v1alpha1
blockOwnerDeletion: true
kind: GrafanaAgent
name: loki
uid: ce95503e-7350-4717-905a-0907a918f413
resourceVersion: "479732"
uid: ceeea6ab-ec5d-4d53-81e8-01abd2c36e5e
type: Opaque
This can help too
type: s3
s3:
endpoint: s3.eu-west-2.amazonaws.com
region: eu-west-2
accessKeyId: ${LOKI_AWS_ACCESS_KEY_ID}
secretAccessKey: ${LOKI_AWS_SECRET_ACCESS_KEY}
kubectl create secret generic loki-aws-secrets --namespace Namespacename --from-literal=LOKI_AWS_ACCESS_KEY_ID= --from-literal=LOKI_AWS_SECRET_ACCESS_KEY=
the suggested solution does not seem to work, still getting a config error about closing braces. hard coding the access/secret key works, so something about parsing the env variable isn’t working
It seems that the error with ‘failed parsing config: missing closing brace’ message is occured when there are dashes in the name of the env variable.
So this config is not working for me
s3:
endpoint: "${GRAFANA-LOKI-S3-ENDPOINT}"
accessKeyId: "${GRAFANA-LOKI-S3-ACCESKEYID}"
secretAccessKey: "${GRAFANA-LOKI-S3-SECRETACCESSKEY}"
and this is working
s3:
endpoint: "${GRAFANALOKIS3ENDPOINT}"
accessKeyId: "${GRAFANALOKIS3ACCESKEYID}"
secretAccessKey: "${GRAFANALOKIS3SECRETACCESSKEY}"
lindhe
December 10, 2024, 10:33am
12
1 Like