Hello everybody,
i’m trying to send live data from telegraf to grafana but these channels do not show up when i try to browse “-- Grafana --” datasource live measurements.
What i find really strange is that logs seem to tell everything is fine.
telegraf log:
2024-10-11T15:05:21Z D! [outputs.http] Wrote batch of 158 metrics in 35.795048ms
2024-10-11T15:05:21Z D! [outputs.http] Buffer fullness: 0 / 10000 metrics
2024-10-11T15:05:23Z D! [outputs.http] Buffer fullness: 0 / 10000 metrics
2024-10-11T15:05:24Z D! [outputs.http] Wrote batch of 1 metrics in 30.05315ms
2024-10-11T15:05:24Z D! [outputs.http] Buffer fullness: 0 / 10000 metrics
2024-10-11T15:05:27Z D! [outputs.http] Buffer fullness: 0 / 10000 metrics
2024-10-11T15:05:30Z D! [outputs.http] Wrote batch of 5 metrics in 29.06532ms
2024-10-11T15:05:30Z D! [outputs.http] Buffer fullness: 0 / 10000 metrics
2024-10-11T15:05:31Z D! [outputs.http] Wrote batch of 93 metrics in 32.981066ms
grafana log:
logger=live.memoryframecache t=2024-10-11T17:05:31.371869381+02:00 level=debug msg="Cache update" orgId=2 channel=stream/telegraf/cpu length=1787
logger=live.managed_stream t=2024-10-11T17:05:31.371895739+02:00 level=debug msg="Publish data to channel" channel=stream/telegraf/cpu dataLength=761
logger=live.memoryframecache t=2024-10-11T17:05:31.371995071+02:00 level=debug msg="Cache update" orgId=2 channel=stream/telegraf/interrupts length=3763
logger=live.managed_stream t=2024-10-11T17:05:31.37203076+02:00 level=debug msg="Publish data to channel" channel=stream/telegraf/interrupts dataLength=3198
logger=live.memoryframecache t=2024-10-11T17:05:31.372091592+02:00 level=debug msg="Cache update" orgId=2 channel=stream/telegraf/soft_interrupts length=1343
logger=live.managed_stream t=2024-10-11T17:05:31.372118379+02:00 level=debug msg="Publish data to channel" channel=stream/telegraf/soft_interrupts dataLength=773
logger=live.memoryframecache t=2024-10-11T17:05:31.372176514+02:00 level=debug msg="Cache update" orgId=2 channel=stream/telegraf/netstat length=1405
logger=live.managed_stream t=2024-10-11T17:05:31.372202494+02:00 level=debug msg="Publish data to channel" channel=stream/telegraf/netstat dataLength=112
logger=live.push_http t=2024-10-11T17:05:33.672900955+02:00 level=debug msg="Live Push request" protocol=http streamId=telegraf bodyLength=225 frameFormat=labels_column
logger=live.memoryframecache t=2024-10-11T17:05:33.673166+02:00 level=debug msg="Cache update" orgId=2 channel=stream/telegraf/http_response length=784
logger=live.managed_stream t=2024-10-11T17:05:33.673200996+02:00 level=debug msg="Publish data to channel" channel=stream/telegraf/http_response dataLength=171
if i try to manually type the channel path like “stream/telegraf/netstat” grafana enters a sort of an infinite loop while trying to run the query (the icon in the “run query” button keeps rotating and never stops).
I really don’t understand what is going on and logs show no other useful info (grafana log is set to debug).
Anyone has a clue? it looks i’m missing something really obvious…
Grafana 11.2.2, Telegraf 1.32.1
Telegraf config:
[[outputs.http]]
flush_interval = "1s"
url = "https://server01.mydomain.com:9005/api/live/push/telegraf"
data_format = "influx"
# can't see any difference in logs if the following are specified or not
tls_cert = "/etc/telegraf/fullchain.pem"
tls_key = "/etc/telegraf/privkey.pem"
[outputs.http.headers]
Authorization = "Bearer ACCESSTOKEN"
i followed the official guides to create an organization unit and relative access token and to configure telegraf as well:
API Tutorial: Create Service Account tokens and dashboards for an organization | Grafana documentation
Stream metrics from Telegraf to Grafana | Grafana Labs
thanks in advance!