Filtering snmp output in influxdb using regex to produce graph

Hi,

I have set up an snmp string to query a device to obtain temperature data.
At this moment, I am only able to view the data in table form. I suspect this is because the snmp return data is returned as such
Column 1: “Date and time”
Column 2: “Ambient Temp, Sideplane: 30 C 86.00F”

I believe that if I would like to display the temps in a graph form, I need to somehow filter/transform the output of the snmp string into just the temp in Celsius. I read numerous posts here that point to regex101 and using Golang for influxdb. However, nothing that I’ve tried that seems to work on regex101 does what I want when I put it in grafana.

I tried to use Transform > Filter by Name > regex pattern >

([0-9][0-9] C)
(\w\w C)

Can someone assist?

grafana version 8.4.3 with influxDB on Ubuntu 20.04.

Welcome @mol3cul3

I am no regex ninja, but how about this?

/(\d+\s*C)/
  • (\d+): Matches one or more digits.
  • \s*: Matches zero or more whitespace characters (including spaces).
  • C: Matches the letter “C” indicating Celsius.

Hi @grant2

Thanks for your suggestion. Again, that seems to work in regex101 but not in grafana. Not sure if I am doing something wrong. With that pattern, it simply says “No data”.
Should I be using this?

Transform > Filter by Name > regex pattern > and stick in the query there as (\d+\s*C)

This is the area where I was thinking you could use Regex. There is a help link as well.

Thanks @grant2 . Unfortunately, that too did not work.
Interestingly, if I put yours or my query into regex101, it targets the value I want to obtain.
However, if I use the substitution, it shows the full string once again.

I found this post which seems to be similar to what I am trying to achieve.
https://community.grafana.com/t/how-to-rename-values-in-grafana-using-regex-based-value-mapping/53616/4
But again, nothing is being filtered nor transformed on my end.

@mol3cul3 to make sure I understand, do you want to capture just the number portion of the celsius temperature?

The substitution will use what you captured with your regex. To get only the celsius temperature I used three capture groups to capture the parts of the line before and after the characters that you want to isolate. Then substituted only the second capture group. I used a field override, so you can choose which field you want to apply the regex to:

In regex101 it looks like this:

Hi @melori.arellano

Thank you. That query enabled me to grab the number portion of the celsius temp!
Please excuse me as I am fairly new to interpreting regex since there are so many expressions that would give you what you want so I’m just trying to understand more.

For example, @grant2 provided this expression (\d+\s*C), which in regex101 works, but I suppose the substitution portion does not reflect this. Is it due to “capture groups” or lack thereof?

Secondly, now that I can get the temperature in table form, how can I convert this into a bar or graph showing the temperature and hostname?
My query itself is simple (perhaps too simple)

SELECT “Temp-Sideplane1” FROM “snmp” WHERE (“agent_host” = ‘ABC’)

and this is what I see now; in table form

Trying to select any other option i.e. Bar gauge or Graph results in “No Data”

I’m not a regex expert either, what I noticed was that the substitution was re-inserting the capture group back into the line in the same position we captured it from.

My workaround was to capture the whole line into three capture groups so there wasn’t any part of the line left, then only substitute back the capture group that we wanted to include.

My guess is that you can’t use timeseries or a graph because they require a numeric field and your temperature is still a string datatype and needs to be changed to a numeric type. You can do that with a transform.

I found a second alternative solution to regex using transform that you can also try:

Use the Extract fields transform and choose the Key+value pairs format to extract the sideplane temperature and dates out of the string. Don’t worry about the extra fields that it finds.

Add a second transform Convert field type to set the extracted fields to the correct datatypes which are time and number in my example.

Now you should be able to use the timeseries visualization:

Hi @melori.arellano

Thanks again for the extremely useful tips. So I tried 2 things, the first was converting the field to Numeric with the previous Value mapping regex query you shared. With that, no value show up in every row.

Next, I tried to use the simpler Extract fields transform function that you shared. I suspect it may be my (older) grafana version but that feature shows up as being Alpha and it does something strange with the extract. If I hit the debug option, the fields show up as Array.

transformations:Array[71]
fields:Array[35]
0:Object
1:Object
2:Object
3:Object
4:Object
5:Object

I’ll keep at it.

@mol3cul3 an interesting thing we’ve noticed is that field overrides and transformations can’t be mixed.

Something else to try - if you want to change the field type with a transform, you’ll want to also use the regex transformation that Grant originally recommended with the same regex that is working for you in value mapping - just remove it from field overrides and add it to the regex transformation.

Thanks @melori.arellano .

Unfortunately, adding it under Transform>Rename by regex didn’t transform anything :blush:

Did you remove the field override? Transform won’t do anything if there is a field override

Why not fix the problem at the source by putting clean data into influxdb?

datetime: 2023-06-13 14:09:15
temptype:Ambient Temp
celsius :30
fahrenheit:86

:point_up:t6: clean. not sure what Sideplane is.

line protocol

sensors,tag1=1 temptype="ambient",celsius=-17.22,fahrenheit=1i 1689675205000000000
sensors,tag1=2 temptype="ambient",celsius=-16.67,fahrenheit=2i 1689675145000000000
sensors,tag1=3 temptype="ambient",celsius=-16.11,fahrenheit=3i 1689675085000000000
sensors,tag1=4 temptype="ambient",celsius=-15.56,fahrenheit=4i 1689675025000000000
sensors,tag1=5 temptype="ambient",celsius=-15.00,fahrenheit=5i 1689674965000000000
sensors,tag1=6 temptype="ambient",celsius=-14.44,fahrenheit=6i 1689674905000000000

regex is good but the minute your data somehow changes the whole thing is guaranteed to be hosed. fix the data would be my recommendation

Then it is cleaner query as well