Grafana and Observium

Grafana Configuration
1

  • What Grafana version and what operating system are you using?
    Latest Grafana and RHEL 8.9

  • What are you trying to achieve?
    Trying to get Observium and Grafana to work together

  • How are you trying to achieve it?

I have setup the Grafana-RRD service from the GitHub - doublemarket/grafana-rrd-server: A HTTP server that helps Grafana read RRD files via Grafana Simple JSON Datasource plugin

I have created a service on my RHEL box that auto starts this at boot. Here is my command line:
grafana-rrd-server -s 300 -r /opt/observium/rrd/

I have configured Grafana to use the Infinity and outdated JSON Plugin. When I ask Grafana to check the datasource, it says its green/ok but no sources are available.

Still not sure if its working. Is there anyone here that has set this up and can offer some advice?

  • What happened?
    I have no idea if it works. Grafana says the datasource is valid but no sources are listed to create dashboards.

  • What did you expect to happen?
    I expected to see sources I can create dashboards from listed in the drop down list from the RRD server data source.

2

what endpoint url did you point grafana to?

is it a GET?

3

I set the URL in the healthcheck window of the Infinity plugin and hit “Test&Save”. The response was:

health check successful with url http://10.1.1.1:9000. http status code received: 200

When I try to create a Dashboard, there are a number of options under the Query(1) tab. I can see that it is set to GET, but I am not sure of the syntax of the URL. I have just used the one from above (http://10.1.1.1:9000). Do I need to add additional information to the URL for this to work?

When I run the GET query from that URL, all I get is “hello”.

4

what is the endpoint that will bring back the data you want to visualize? Pretty sure it is not just hello :wink:

5

Hello is the default to show that the port is open. I guess what I am missing is how to formulate the query.

I tried this and did not return anything other than hello.
http://10.1.1.1:9000/cisco-switch-2

I have a corresponding RRD folder under /opt/observium/rrd with that name. It still returned a hello and nothing else.

The documentation on GitHub for this server is very limited. They do not list any query examples so I am in the dark as to what this server is expecting from the Infiniity plugin. But I am sure its just the syntax of the query that I am missing.

6

http://10.1.1.1:9000/search

what if you tacked on the /search endpoint what do you see?

7

no data.

Obviously I am new to this. Observium and Grafana have been dropped in my lap with the ol “just figure it out” mantra…

Any suggestions anyone might have on how to get Grafana and Observium to work together is much appreciated.

Thank you for the help so far.

1 Like
8

so you do have files under /opt/observium/rrd/ ?

10

Absolutely! Observium is running and logging just fine. It is creating the RRD files in individual folders named as the logged device under /opt/observium/rrd

11

this is what I see in the github repo grafana-rrd-server/rrdserver.go at master · doublemarket/grafana-rrd-server · GitHub

	SetArgs()

	http.HandleFunc("/search", search)
	http.HandleFunc("/query", query)
	http.HandleFunc("/annotations", annotations)
	http.HandleFunc("/", hello)

so not sure why /search shows nothing

12

Just ran the query with the console on the observium server open.

When I run the query request, the console reports:

ERROR: Cannot decode the request
EOF

Looks like the grafana-rrd-server is not happy with the request…

1 Like
13

what does the content of an rrd file look like? maybe you can use nodejs maybe to get what you need or some other method to serve the content of the files.

14

I’m looking into this. Thanks for the help and the advice!

15

The grafana-rrd-server is the only option that allows for scanning of the rrd files. All the other options require a manual config for each device that is monitored. If you are using Observium in a small environment, then this might work. My environment will be monitoring over a hundred different devices across multiple departments. So, its not going to work if I need to create a custom PHP or Node.js config for each device. It is going to be an administrative nightmare.

Any other ideas on how to get Observium and Grafana to work together?

1 Like
17

This was AWESOME! I am actually getting data from my Observium. I was able to get it to pull via HTTPS which is huge!

The problem now is that I need to format it!

Any pointers on Formatting? I’ll open another forum post if needed.

Thanks!

1 Like
18

cant see your data remotely on your pc, I usually have that jedi power but not today

19

Use the force… OBIWAN!! :slight_smile:

Screenshot 2024-08-14 at 10.21.56 AM
Screenshot 2024-08-14 at 10.21.56 AM3116Ă—1154 186 KB

This is what I am getting from the Observium API. Its just string data with a “:” delimiter when i query /api/v0/devices/ and api/v0/devices/?os=ios.

Not sure how to get it parsed into columns or something that Grafana can actually display. I played around with the different type/parser options but did not make much headway.

Is it time for me to RTM? Do I need to create a custom filter someplace?

Thanks for all the help!!

20

:smile:you want me to retype the data from the image you posted?

21

Sorry… I’m an idiot… I won’t post the entire output since there is a LOT. I have posted the first device. This came out as a single line. A very LONG single line.

{“2”:{“agent_version”:null,“arch”:null,“asset_tag”:null,“bgpLocalAs”:null,“device_id”:“2”,“disabled”:“0”,“distro”:“Ubuntu”,“distro_ver”:“”,“features”:“Ubuntu”,“force_discovery”:“0”,“hardware”:“Generic x86 [64bit]”,“hostname”:“stratus.example.c0m”,“icon”:null,“ignore”:“0”,“ignore_until”:null,“ip”:“10.1.1.24”,“kernel”:“5.15.0-88-generic”,“label”:null,“last_alerter”:“2024-08-14 17:35:03”,“last_discovered”:“2024-08-14 13:33:20”,“last_discovered_timetaken”:“2.84”,“last_polled”:“2024-08-14 17:35:02”,“last_polled_timetaken”:“0.55”,“last_rebooted”:“1717603677”,“location”:“Unknown (edit /etc/snmp/snmpd.conf)”,“location_city”:“Unknown”,“location_country”:“Unknown”,“location_county”:“Unknown”,“location_geoapi”:“geocodefarm”,“location_id”:“2”,“location_lat”:null,“location_lon”:null,“location_manual”:“0”,“location_state”:“Unknown”,“location_status”:“Geocoding ENABLED, try detect device coordinates:\n by FORWARD query (API: GEOCODEFARM, sysLocation: Unknown (edit /etc/snmp/snmpd.conf)) - \n GEO API REQUEST: Announcements - Geocode.Farm 11:25:02”,“os”:“linux”,“poller_id”:“0”,“purpose”:null,“serial”:null,“snmpEngineID”:“”,“snmp_maxrep”:null,“snmp_port”:“161”,“snmp_retries”:null,“snmp_timeout”:null,“snmp_transport”:“udp”,“snmp_version”:“v2c”,“snmpable”:null,“ssh_port”:“22”,“status”:“1”,“status_type”:“ok”,“sysContact”:“Root root@localhost (configure /etc/snmp/snmp.local.conf)”,“sysDescr”:“Linux node0 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64”,“sysName”:“node0”,“sysORID”:“["TCP-MIB","UDP-MIB"]”,“sysObjectID”:“.1.3.6.1.4.1.8072.3.2.10”,“type”:“server”,“uptime”:“6053225”,“vendor”:null,“version”:“5.15.0-88-generic”},

22

again, you want me to clean up the data for you? Clean it up and post valid json

anyways you can use infinity plugin to parse it using UQL query language for jsonata

https://grafana.github.io/grafana-infinity-datasource/docs/uql