Secure loki and promtail via TLS

I’m trying to establish a secure connection via TLS between my promtail client and loki server. I am unable to figure out how to make this happen. It appears I’m able to get promtail configure to send content via TLS with the below block within the config file. However when I try to configure loki for TLS I’m hitting a road block, and I’m unable to find the documentation stating how to.
Promtail snip it that seems to be working:

  • url:
    batchwait: 1s
    batchsize: 102400 # ~100KB
    ca_file: /etc/loki/certs/myCA.pem
    cert_file: /etc/loki/certs/loki-local.crt
    key_file: /etc/loki/certs/loki-local.key

Loki attempts.
ExecStart=/usr/local/bin/loki -config.file /etc/loki/loki-config.yaml -server.http-tls-ca-path /etc/loki/certs/myCA.pem -server.http-tls-cert-path /etc/loki/certs/loki-local.crt -server.http-tls-key-path /etc/loki/certs/loki-local.key
other various configuration changes within the yaml config, but non have worked. From what I’ve read setting this in systemd is proper, but the error I get is below.
caller=log.go:106 msg=“error running loki” err="error generating http tls config: Client CA’s have been configured without a Client Auth Policy\nerror initialising module: server\ngith>

Any help is appreciated.

1 Like

how to run loki over https (TLS) via the configuration settings that go in local-config.yaml …in this example, the endpoint is using 8443:

  http_listen_port: 8443
  http_tls_config: &tls_server_config
    cert_file: /etc/loki/myserver.cert.pem
    key_file: /etc/loki/myserver.key.pem

this is an alternative to using a reverse proxy.
The above is based on the fact that loki uses which in turn uses

Hi all,

I could configure TLS in Promtail and Loki, but now the communication between them does not work.

Configurations for Loki:

    config: |
      auth_enabled: false
        http_listen_port: 3100
        http_tls_config: &tls_server_config
          cert_file: /etc/loki/cert.pem
          key_file: /etc/loki/key.pem

Configurations for Promtail:

  uri: http://<Redacted>/loki/api/v1/push
    ca_file: /etc/loki/certs/ca.crt
    cert_file: /etc/loki/certs/cert.pem
    key_file: /etc/loki/certs/key.pem

  - name: inf-secret
      secretName: obs-secret
      optional: false

  - name: inf-secret
    mountPath: "/etc/loki/certs"

I am using secrets to store the certificates and the services can find them. But I am getting the following errors:

Loki Distributed:

ts=2022-08-12T13:52:30.424816874Z caller=memberlist_logger.go:74 level=warn msg="Failed to resolve obs-loki-distributed-memberlist: lookup obs-loki-distributed-memberlist on no such host"


level=warn ts=2022-08-12T14:00:25.150418634Z caller=client.go:369 component=client msg="error sending batch, will retry" status=-1 error="Post \"\": dial tcp connect: connection refused"
level=warn ts=2022-08-12T14:01:21.774977915Z caller=client.go:369 component=client msg="error sending batch, will retry" status=-1 error="Post \"\": dial tcp connect: connection refused"

PS: I am configuring through Helm charts.

Hi Zach,

Have you tried the ExecStart parameter -server.http-tls-client-auth=RequestClientCert? according to the weaveworks/weave code in GitHub, these are the possible values:

  • “RequestClientCert”
  • “RequireAnyClientCert”
  • “VerifyClientCertIfGiven”
  • “RequireAndVerifyClientCert”
  • “” or “NoClientCert”

I checked the code here: tls_config.go.
PS: Another useful link

@ancarvalho it looks like your configuration for promtail specifies that the client uri should use http in lieu of https which is probably the cause of your connection refused in the promtail logs.