Grafana Timestream Data Source

Hello,

I am using Grafana to display data from AWS Timestream, it seems to be very slow to display the data, I have tried changing some datasource configuration but the situation remains the same, it is really very slow, do you have any ideas to improve performance ?

In the first picture you can see the dashboard loading, it stays loading for several minutes.

In the second pic you’ll find the datasource configuration

Opened a Bug in Grafana Timestream Plugin GitHub prj : Slowness to retrieve data in dashboard ( POST calls fail repeatedly ) · Issue #305 · grafana/timestream-datasource · GitHub

how long does that exact query take outside of grafana ie a timestream query tool

can you share the query?

The query extracted from the panel and run on the timestream query editor takes 2 sec (see image)

Reloading graphana after a few minutes, it started the slow loading again, and opening Chrome’s DevTool you can see how the query calls repeatedly fail until an answer comes after several minutes (as you can see in the second pic)


1 Like

so this might indicate an issue with that data source. All the errors are POSTs . why is it trying to POST when fetching data?

might want to post this in grafana github

I don’t know why, but it seems to use the POST call with the query used in the payload.

OK I’ll open a Bug in the github repository

1 Like

your try_cast stuff is worrisome. also do you have index on time measure_name and asset_code columns?

i’ve used try_cast because not all values are integer inside the Timestream table (and it’s useful to display those type of data in Timeseries visualization) and no, i don’t have indexes for those columns (dimensions). At least with also some simple queries the situation remains the same

Opened a bug into Grafana Timestream Plugin : Slowness to retrieve data in dashboard ( POST calls fail repeatedly ) · Issue #305 · grafana/timestream-datasource · GitHub

1 Like

yeah that was just some side points but worth looking into. can you try that same query without the use of try_cast
you should really not be plotting any values that are not int though right? also it raises a lot of ??s as to why that one column has 2 types of data types. does not pass the sniff test :blowfish:

Without the use of try_cast the situation remains the same, it seems that the authorization provider via SDK takes a long time before it has a token to be able to query AWS services and every time it expires it takes the same amount of time to request it

2 Likes

does timestream provide a rest api? wonder if that is faster

1 Like

From Slowness to retrieve data in dashboard ( POST calls fail repeatedly ) · Issue #305 · grafana/timestream-datasource · GitHub I opened it seems to be strictly related to the Authentication Provider (SDK Default) because by selecting the method via credentials everything works immediately, even heavier queries are executed quickly

1 Like