Extract values from application logfile and display them on Grafana Cloud dashboard

Good day, I have already read the documentation and tried to find some examples that I can re-use but without success. Grot AI could not give me a proper answer either.
Also, I have to say I am not a programmer and so I do not understand everything, but I guess and hope the my use case could be realized with Grafana services like Alloy and Loki.
My technical setup is as follows.
A home automation server is running on a Raspberry Pi.
Several devices like thermostats are connected to the radio module on the Raspberry Pi.
Each device writes its status and values in a dedicated log file. Every month a new log file is created.
The log files are all in the same directory /opt/fhem/log
Here comes an example for the 10 last lines in the log file of thermostat called “HmIP_WTH_2_000A9F29899C87”.

pi@raspi4:/opt/fhem/log $ tail -10 HmIP_WTH_2_000A9F29899C87-2025-05.log
2025-05-05_15:12:56 HmIP_WTH_2_000A9F29899C87 measured-temp: 21.5
2025-05-05_15:12:56 HmIP_WTH_2_000A9F29899C87 HUMIDITY: 40
2025-05-05_15:12:56 HmIP_WTH_2_000A9F29899C87 humidity: 40
2025-05-05_15:12:56 HmIP_WTH_2_000A9F29899C87 hmstate: 21.5
2025-05-05_15:55:50 HmIP_WTH_2_000A9F29899C87 HUMIDITY: 41
2025-05-05_15:55:50 HmIP_WTH_2_000A9F29899C87 humidity: 41
2025-05-05_15:55:50 HmIP_WTH_2_000A9F29899C87 21.6
2025-05-05_15:55:50 HmIP_WTH_2_000A9F29899C87 ACTUAL_TEMPERATURE: 21.6
2025-05-05_15:55:50 HmIP_WTH_2_000A9F29899C87 measured-temp: 21.6
2025-05-05_15:55:50 HmIP_WTH_2_000A9F29899C87 hmstate: 21.6

Logically, the lines consist of 3 columns.
First column: date and time like 2025-05-05_15:55:50
Second column: device name like HmIP_WTH_2_000A9F29899C87
Third column: key value pair like ACTUAL_TEMPERATURE: 21.6
Behind the key is a colon, then comes a blank and at the end the value.
The value can have one place after the decimal point, as for the temperature, or has no decimal point, as for the humidity.

Now I want to bring especially the temperature and the humidity on two separate dashboards in Grafana Cloud.
The x-axis should contain the time from the log file (not the time when the data is reaching Grafana Cloud), the y-axis the value. Finally, I would like to have a graph showing the measured temperature and humidity over the time.

The Grafana Alloy service is already installed and running on the Raspberry Pi.
My first config.alloy file looks like this.

$ sudo cat /etc/alloy/config.alloy
 local.file_match "fhemlogs" {
     path_targets = [{"__path__" = "/opt/fhem/log/*.log"}]
     sync_period = "5s"
 }


  loki.source.file "log_scrape" {
    targets    = local.file_match.fhemlogs.targets
    forward_to = [loki.process.filter_logs.receiver]
    tail_from_end = true
  }

  loki.process "filter_logs" {
    stage.drop {
        source = ""
        expression  = ".*Connection closed by authenticating user root"
        drop_counter_reason = "noisy"
      }
    forward_to = [loki.write.grafanacloud.receiver]
    }

loki.write "grafanacloud" {
  endpoint {
    url = "https://logs-prod-012.grafana.net/loki/api/v1/push"

    basic_auth {
      username = "1152179"
      password = "my_token"
    }
  }
}

In Grafana Cloud under Drilldown / Logs I see the logs coming in but the only service is “unknown”, see screenshot.
How must the config.alloy be designed in order to solve my challenge and bring the dedicated values like temperature and humidity on Grafana Cloud dashboards?

Many thanks in advance for any hints and suggestions!

Not sure I quite understand your question, what part of the log are you looking to set as service? If it’s the second part (device name, in your example), then you’d use loki.process, first regex the second part of the log, then you can set it as a label.

1 Like

Hello,

Im not sure I fully understand your need. If you want to get the value of each “label” (temp, humidity) use this query:

{job="your-job-name"} |= `measured-temp_OR_humidity` | pattern `<TIME> <_> <_> <VALUE>`

After that go to the transformations section,

Extract the labels to fields.

and change the fields’ data types.

Thanks for the hint! I will try it out.