Use unit label to format value labels

Grafana Loki
,
1

I am currently trying to extract data from a text log:

Backup created.
------------------------------------------------------------------------------
Chunk index:                  510649             11946505
                       Unique chunks         Total chunks
All archives:               14.57 TB             14.12 TB            744.29 GB
This archive:              851.18 GB            823.80 GB              2.60 GB
                       Original size      Compressed size    Deduplicated size
------------------------------------------------------------------------------

I am interested in viewing the deduplicated_size in GB (this should result in “744.29 GB” for this example)
With a regex query, I am able to extract the value (744.29) and view it in a time series:

max by (label_name) (max_over_time({$label_name=~"$label_value", job=~"$job", instance=~"$instance"} |= `All archives: ` | regexp `(?P<archive_type>All archives|This archive):\s+(?P<original_size>\d+\.\d{2}) (?P<original_unit>PB|TB|GB|MB)\s+(?P<compressed_size>\d+\.\d{2}) (?P<compressed_unit>PB|TB|GB|MB)\s+(?P<deduplicated_size>\d+\.\d{2}) (?P<deduplicated_unit>PB|TB|GB|MB)` | unwrap deduplicated_size [$__interval]))

However, I could not use the extracted label deduplicated_unit to map a unit to deduplicated size.
Do Grafana or Loki have options to map these units? Or are there workarounds to view the data properly?
Thanks for any help on this issue, it bothered me for several days already!

2

I don’t believe you can dynamically set unit on a Grafana Graph. Your goal is to have an unified unit (byte), and ideally your log should have a unified unit (one should always log with analytics in mind whenever possible), but if that’s not possible then you’ll have to do some processing loki query. Something like this (not tested):

{SELECTOR}
  | regexp `(?P<archive_type>All archives|This archive):\s+(?P<original_size>\d+\.\d{2}) (?P<original_unit>PB|TB|GB|MB)\s+(?P<compressed_size>\d+\.\d{2}) (?P<compressed_unit>PB|TB|GB|MB)\s+(?P<deduplicated_size>\d+\.\d{2}) (?P<deduplicated_unit>PB|TB|GB|MB)`
  | line_format `deduplicated_size_byte={{ if eq .deduplicated_unit "MB" }}{{ mulf .deduplicated_size 1024 1024 }}{{ else if eq .deduplicated_unit "GB" }}{{ mulf .deduplicated_size 1024 1024 1024 }}{{ else if eq .deduplicated_unit "TB" }}{{ mulf .deduplicated_size 1024 1024 1024 1024 }}{{ end }}`
3

Thank you very much! I made it work with max by (label_name) (max_over_time({$label_name=~"$label_value", job=~"$job", instance=~"$instance"} |= `All archives: ` | regexp `(?P<archive_type>All archives|This archive):\s+(?P<original_size>\d+\.\d{2}) (?P<original_unit>PB|TB|GB|MB)\s+(?P<compressed_size>\d+\.\d{2}) (?P<compressed_unit>PB|TB|GB|MB)\s+(?P<deduplicated_size>\d+\.\d{2}) (?P<deduplicated_unit>PB|TB|GB|MB)` | line_format `{{ if eq .deduplicated_unit "MB" }}{{ mulf .deduplicated_size 1024 1024 }}{{ else if eq .deduplicated_unit "GB" }}{{ mulf .deduplicated_size 1024 1024 1024 }}{{ else if eq .deduplicated_unit "TB" }}{{ mulf .deduplicated_size 1024 1024 1024 1024 }}{{ else if eq .deduplicated_unit "PB" }}{{ mulf .deduplicated_size 1024 1024 1024 1024 1024 }}{{ end }}` | pattern `<deduplicated_size_byte>` | unwrap deduplicated_size_byte [$__interval]))

4

Capture both the size and unit into a single label and use the bytes function, see Metric queries | Grafana Loki documentation