Grafana is not showing actual data points when we have prometheus as data source

Grafana Prometheus
1

Hi,

Basically, we have networking devices which will stream every 1 second to our collector. We process it and convert it into prometheus metric format and send it via push gateway. We are scraping the metrics every 250ms on prometheus job… When we look at records on prometheus console we have the messages with value along with the right timestamp.

But, when we plot a graph on Grafana: we are not seeing the actual timestamp of message on each datapoint. Data points are getting replicated in because of step interval and resolution. Though we set 1 sec as scraping interval on grafana and 1/1 as resolution with 1sec as step interval. But when we look at data points, its showing every second through there are few dropped or missed collection from networking device(which is streaming the data). Basically, it is happening for all kinds of data which we store it on prometheus.

The no.of data points on prometheus console and grafana are different for the same query it is because of step interval and resolution.

Can you please help us to get only the actual timestamped data points on grafana.

Regards,
Rajesh

2

Not sure this is possible with Prometheus, the step parameter cannot go below 1s last time I checked.

3

@torkel Minimum step param 1 sec should be fine for us. But the actual data points on prometheus and grafana are different. Is there anyway we can just plot the graph with actual series without any additional points which are getting added because of step interval and resolution params.

for ex: If we are getting 8 data points in last 10 seconds then we have to show only 8 instead of 10 or more on grafana as well.

4

When using the table panel, you can show the raw points from Prometheus by checking the “Instant” box and using a range query. For example, name{label1=value1}[10m]

1 Like