CSV as DataSource

Hi-

I have a datasource that may be helpful:
https://github.com/ryantxu/file-system-datasource
It is a bit too complicated since it tried to support many file types. CSV is the one that works best though :slight_smile:

In grafana core, if you enable ‘alpha’ panels, you will see a datasource named ‘Input’ – this lets you save CSV data in the datasource or panel.

I hope to make using simple files part of grafana core, but currently it requires a plugin.

@pooh – you may not need to query it. Really depends what is in the file and how big it is. With the above datasource, the file name represents a time span and that is really the query

ryan

1 Like

Thanks so much @ryantxu

I need to show someone dashboard from QlikView on TV and i think that best choice is Grafana, because it’s free and works perfectly

Yes Grafana is a good option, but what about import csv to influxdb ? :

A another option is to import csv to influxdb, personally i use this : https://github.com/fabio-miranda/csv-to-influxdb

and my command line to import a csv file :

python csv-to-influxdb.py --dbname db_name --input /tmp/3.csv --tagcolumns location --fieldcolumns Température --timeformat '%Y-%m-%dT%H:%M:%SZ' --metricname v2

Some time ago, I created a simple Node.JS service called CSVServer, especially for parsing CSV files and sending the results to Grafana via the SimpleJSON data source.

Take a look at this:
https://github.com/michaeldmoore/CSVServer

It’s much, much easier than messing with an influx or graphite database.

It supports table and time series Grafana data formats and works pretty well for small-ish CSV files (say a few hundred rows or more). I haven’t put any caching code in yet, so it will get bogged down if you try to parse megabytes of data - every query causes the file to be reparsed too, so it will have performance issues if you have hundreds of simultaneous users refreshing dashboards every few seconds.

Right now, it doesn’t use the dashboard start and end time filters either, so this might be a problem, depending on you CSV data files. I have an update in the works to add time filtering but I don’t want to check that bit of code in yet - not until I figure out some caching mechanism.

Check out the READ.ME section in github and see if this works for what you need. I expect it will.

Michael Moore

1 Like

Hi,
You can try this one csv plugin
It is a native Grafana Datasource plugin (written in golang) which supports data filtering

1 Like

Hello @paveldanilin

This is really cool. :clap: :clap:
But, I tried to install csv plugins but it didn’t work for me. Any suggestions?
Grafana is installed on windows server 2016.
So, what I have done is download the github file “grafana-csv-plugin-master” and put this into “C:\Program Files\GrafanaLabs\grafana\data\plugins\grafana-csv-plugin-master” directory.
then restart the grafana service.

Hi @saminda
It seems I forgot to add the “dist/” folder which holds binaries to the plugin repo.
I have just fixed it
Could you repeat all steps? (download; unpack to grafana/plugins/; restart grafana service)
If you need a binary file for another platform (linux / darwin) let me know I will try to build it.

@paveldanilin
Thanks !
I have replaced the folder with new downloaded file.
But, still no luck.

Hi @saminda
Any errors in grafana.log?
Please, set up the log level as debug and try to check for errors.
In a normal situation, there should be a record

t=2020-05-20T17:06:37+1000 lvl=dbug msg=“plugin started” logger=plugins.backend pluginId=grafana-csv-plugin path=“C:\Program Files\GrafanaLabs\grafana\data\plugins\grafana-csv-plugin-master\grafana-csv-plug
in-master\dist/grafana-csv-plugin_windows_amd64.exe” pid=1700

Could you share with me your grafana.log? (Just send it to me by email)

1 Like

Hi!
I have just added executable for Linux you can find it in the repo.
I also added info about the error that you had faced and how to fix it.
Thank you for testing the plugin!

1 Like

Thanks, I will check that also.
This is really helpful plugin. I really appreciate it for sharing with the public. :clap:

1 Like

This plugin looks good awesome and i was trying to set it up in my windows local machine. I gave the path as my window dir - D:\xxx.csv, but it says validation error.

Is it like it works only on linux machine?

Log:
t=2020-06-23T06:56:43-0400 lvl=eror msg=“Validation error, invalid URL: “D:\\alertisite\\hsm_stats.csv”” logger=context userId=1 orgId=1 uname=admin error=“Validation of data source URL “http://D:\\alertisite\\hsm_stats.csv” failed: parse “http://D:\\alertisite\\hsm_stats.csv”: invalid port “:\\alertisite\\hsm_stats.csv” after host” remote_addr=[::1]

That seems unlikely given that there is a Windows executable in the
repository.

Antony.

Hi!
The plugin works fine with Grafana 6.7.4 (linux/win)
As far as I can see at source code, starting from Grafana 7.x.x there is a validation for the config form and I must patch it to get it to work with 7.x.x.
Could you open an issue in repo?

1 Like

sure bro… good deal

Hey Pavel - Thanks for the fix, its working like a charm now :slight_smile: I was able to add the datasource and it auto detected the column from my sheet.

1 Like

Hi @paveldanilin: We were trying to add the DS using Linux setup, DS added successfully. But while fetching query i am getting error, suspect its not updatng in db properly.

unexpected token Identifier(“hsm_stats”) (1:15) | SELECT * from hsm_stats | …^

Object

status:400

statusText:“Bad Request”

data:Object

results:Object

message:“unexpected token Identifier(“hsm_stats”) (1:15) | SELECT * from hsm_stats | …^”

message:“unexpected token Identifier(“hsm_stats”) (1:15) | SELECT * from hsm_stats | …^”

Did you manage to resolve the issue?
If no, please, open an issue in repo.

Yeah still seeing it and have opened an issue. Thanks :slight_smile:

Gather the latest updated and its working fine now :slight_smile:

Thanks again for quicker turnaround :heart_eyes: :love_you_gesture: :clap: