Ingesting hourly data from a JSON API

I typically use Grafana with a Prometheus and that’s been great / life changing.

Now I want to plot over time a JSON API using Grafana, but Prometheus can’t just ingest the API easily:
https://groups.google.com/forum/#!topic/prometheus-users/s0jzpVHTKUY

I guess it doesn’t make sense to sample an API that’s updated hourly, but more importantly I want to pull out data from 2016-02-10 until present data. And then plot it in Grafana.

So how does one get the data produced by curl -X GET "https://api.data.gov.sg/v1/environment/pm25?date=2016-03-11" -H "accept: application/json" into Grafana? Or rather what’s a suitable data source for this type of ingestion?

Thank you,

I don’t quite understand what you would like to do.
Do you have some external data source you want to plot in Grafana, or does Prometheus generate data you want to plot?

Dear Kai,

I’m assuming you are about to ingest air quality data into Prometheus/Grafana? While we are using InfluxDB for the very same purpose, it might exactly match what we are doing already with the data from https://luftdaten.info/.

So, you might enjoy having a look at Luftdatenpumpe:

Just let us know when you have further questions about it.

Cheers,
Andreas.

Thank you Andreas,

So InfluxDB huh? Just asking them quickly if it’s right for my use case: https://community.influxdata.com/t/json-ingestion-from-an-api/8966

Wish things were simpler. :wink:

Don’t have time to study https://github.com/hiveeyes/luftdatenpumpe

Dear Kai,

Admittedly, Luftdatenpumpe is pretty heavy as it does many things around air quality data ingesting. When looking at the JSON file you referenced in your post, a more basic importer thing would also be feasible as a starter.

Don’t be afraid of the MQTT thing, this is just an option of this program, you obviously might want to focus on the InfluxDB side here.

As you are asking about the optimal database, it’s not that we are saying it is only possible with InfluxDB - it’s just that we have good experiences with it when ingesting non-server related metrics from environmental monitoring, also beyond air quality data, see also

With kind regards,
Andreas.

1 Like

Hi Team,

Recently we have configured Grafana in our account.
After installing we can see only few things. I did some research and imported few dashboards but unable to see which the data i need.

Mainly we are looking for below things.
1)Pods status
Incase of any Pod fails i want to see which Pod failed i.e Pod name from Grafana.
2)Services status in Pod
what is the status of the service like up or down.
for any reason if any service goes down

Need to view in Grafana.

Kindly help on this. I am new to Grafana
Appreciate your help on this.
If you have any dashboard available with above features kindly share the same.

Dear @hendry,

Luftdatenpumpe is now able to process air quality data coming from a 52°North Sensor Observation Service instance of the Belgian Interregional Environment Agency (IRCEL - CELINE). If this sparks your interest, you might also enjoy visiting and exploring the respective staging system at https://vmm.hiveeyes.org/.

Getting from one to two data sources here was the most important thing the internal architecture needed before thinking about adding even more data sources. So, Luftdatenpumpe should now be ready for adding data from Singapore’s air quality network you have been referring to the other day.

With kind regards,
Andreas.