Issue with grafana graphite database and graphics metrics

kawless · February 5, 2018, 10:29pm

Hi there,

I have an issue when I try to create a data source and then viewing the graphics associated with this datasource. I’m trying to create a Graphite database and I’m not using the port 8080 but the 8085 one.
Here’s a screenshot of my configs :

I get an empty red alert box which is hard for me to pin point the error code.
So when I’m about to choose my Graphite database and for instance I try to write “apps” into my query I get this error message :

I also configured my grafana localhost port at 8090 because I’m using 8080 for jenkins. I don’t if that can cause an issue. I appreciate any kind of help.

Thanks in advance!

kawless · February 5, 2018, 10:32pm

Here’s another screenshot that shows the same error code for the number of query’s that you have :

daniellee · February 14, 2018, 4:05pm

Any special reason for choosing Http Setting Access as Direct? You probably want proxy mode.

Direct means directly from your browser to the Graphite database. So you are probably getting a Cross Origin error (which is a security feature built into all modern browsers). Proxy mode adds the appropriate CORS headers.

kawless · February 14, 2018, 4:12pm

Hi daniellee,

Thank you for your answer. Yes I tried proxy mode in this thread : https://community.grafana.com/t/grafana-graphic-database-issue-with-metrics/5323/8

I went in details about what I did in this thread.

But now I have a bad Gateway error :

Here’s the error when I try to create a graph :
That’s the query inspector :

Also I would like to know if it’s possible to use 8085 in the future because I’m using 8080 a lot for other purposes.

Thank you very much.

daniellee · February 14, 2018, 5:00pm

Does the server Grafana is installed on have access to your graphite server? It does not look like it - firewall problems?

I think you can configure the graphite port to be whatever you want:

http://graphite.readthedocs.io/en/latest/config-local-settings.html

kawless · February 14, 2018, 8:28pm

Hi daniellee, thank you for your answer.

Do I need to add rules into the windows firewall system? I’ve been looking at :

github.com/grafana/grafana

Grafana logs says "Database table locked, sleeping then retrying" logger=sqlstore retry=0.

opened 02:57PM - 13 Nov 17 UTC

closed 01:37PM - 14 Nov 17 UTC

ghost

area/backend/db

Hi, I am new to grafana, using it for some POC but ran into the issue. Log s…ays:-- t=2017-11-13T14:51:06+0000 lvl=info msg="Initializing HTTP Server" logger=http.server address=0.0.0.0:3000 protocol=http subUrl= socket= t=2017-11-13T14:56:30+0000 lvl=info msg="Database table locked, sleeping then retrying" logger=sqlstore retry=0 - What Grafana version are you using? Grafana v4.6.1 - What datasource are you using? influxdb - What OS are you running grafana on? Linux a198-18-52-100 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux - What did you do? installed influxdb and grafana was populating some data in DB and trying to see in grafana. For some reason data was not coming checked /var/log/grafana/grafana.log saw the error "Grafana logs says "Database table locked, sleeping then retrying" logger=sqlstore retry=0." - What was the expected result? wanted to see the request and status code for that request - What happened instead? Saw error "Grafana logs says "Database table locked, sleeping then retrying" logger=sqlstore retry=0." Thanks in Advance for the help

github.com/grafana/grafana

"Database table is locked" error

opened 06:12PM - 30 Mar 17 UTC

closed 10:19AM - 15 May 17 UTC

mattttt

type/bug area/backend/db

@woodsaj and I ran into this issue while testing some new worldPing updates. Spe…cifically, the latest version of wP will automatically update your datasources to the latest settings for the app. The requests were being sent concurrently and we saw: ``` EROR[03-30|13:50:29] Failed to update datasource logger=context userId=1 orgId=1 uname=admin error="database table is locked" EROR[03-30|13:50:29] Request Completed logger=context userId=1 orgId=1 uname=admin method=PUT path=/api/datasources/1 status=500 remote_addr=[::1] time_ms=8ns size=41 ``` Rather than making changes in the app itself, @woodsaj noted that this could happen any time concurrent requests needed to modify the same table - Ie saving 2 dashboards at the same time. Is this something that should be addressed in Grafana?

Because when I open my log into grafana I have this :

I don’t know if my issue is related to this.

Thank you.

daniellee · February 15, 2018, 10:35am

The “Database table is locked” message is just a warning and Grafana automatically retries. This is a limitation of using Sqlite as the backend db. If you have lots of people saving dashboards then this could be a problem but then you will see errors and not just the warning. So this has nothing to do with your problem.

A 502 error means that the connection between Grafana and Graphite does not work.

Does the graphite url work? Can you access http://10.10.2.229:8080 from your machine and from the server? (you should come to the graphite browser page)
You can try adding the port in the Windows firewall. This is an old video but I think it is mostly the same procedure: https://www.youtube.com/watch?v=cli-Rke3KYo

kawless · February 15, 2018, 10:25pm

Hi daniellee,

Thank you for clarifying everything. I have a better understanding now.

No the url dosen’t work. I’m not sure if it’s a firewall related issue because
I was running jenkins earlier on the port 8080 and everything was fine. Maybe
it’s not related?

On a side note, do I need to install graphite outside of grafana before using it?
I though that creating graphite within grafana was enough but maybe I missed
a step in the documentation.

In the video, I’ve noticed that the person is running his db as a service like grafana.
I’m not doing that with graphite, do I need to do the same? If so, I will look into it (also
the firewall).

Thanks for all the help so far.

daniellee · February 15, 2018, 10:52pm

So you never installed Graphite? Grafana and Graphite are totally separate products. Grafana is a visualization tool. Graphite is a database.

Native Graphite does not work on Windows so I was wondering how you managed to install it. From the Graphite docs

Unfortunately, native Graphite on Windows is completely unsupported, but you can run Graphite on Windows in Docker or the Installing via Synthesize article will help you set up a Vagrant VM that will run Graphite. In order to leverage this, you will need to install Vagrant.

You should probably look at either InfluxDB or Prometheus for your time series database if you want to monitor a Windows environment. Both work great with Grafana.

kawless · February 19, 2018, 7:41pm

Hi daniellee,

Sorry for the late reply was busy with other things and thank you for your suggestions. I’m now using influxDB and I’m able to use a database.

Downloaded influxDB from their website, started the influxd.exe :

Started a standard console with the folder path to log into my db and typed influx,
CREATE DATABASE influxdb_test :

I linked my influxDB with my grafana and everything seems to be working fine :

but when I try to load a graph model, exemple :

Here’s an exemple :

I have this :

I’m missing all the data from the graph that I imported but it’s possible that
I’m missing some crucial steps.
I was wondering why I don’t have any data? I’ll keep looking in the meaning.

Thanks for all the help Daniellee, I appreciate it!

kawless · February 19, 2018, 7:54pm

Here’s the query :

daniellee · February 19, 2018, 8:27pm

Are you using the ping Telegraf plugin and is it successfully writing data to the influx database?

Telegraf is a collector and collects data from the server you are monitoring and writes it to the InfluxDB database. If you google for a getting started guide, there are loads of blog posts as well as the InfluxDB documentation.

P.S. You can use the InfluxDB Admin plugin to explore your InfluxDB database and/or to insert some test data.

kawless · February 23, 2018, 10:17pm

Hi daniellee,

Thank you for this advice. I forgot avec adding up Telegraf as my main data writer into my InfluxDB. Hence, I’m having trouble setting it up. I tried various ways with no succes but my InfluxDB reacts a certain way when I start my telegraf.exe.

When I try to enter commands in InfluxDB I get this message :

Im using the database that I created : influxdb_test :

My Telegraf folder is located via this path :

I use this command to start my telegraph.exe and I have this as a result :

When I go back to my influxDB.exe, my console is writing POST’s :

Datasource is once again working :

I get this plugin for testing purposes :

I import this dashboard into mine :

But I have no data :

Once again, I’m kind of lost but I’ll keep looking. Maybe I’ll find a way or
I made a mistake in my procedure.

Thank you in advance, one again.

daniellee · February 28, 2018, 11:35am

Looks like the data from the telegraf is being stored in a database called telegraf. And the influxdb_test database is empty I guess? - which is why you have no data in your dashboard.

Can you change your datasource in Grafana to connect to the telegraf database instead of the influxdb_test and test if that works

kawless · March 1, 2018, 9:54pm

Hi daniellee,

Thank you for the suggestion. I changed the configuration of my telegraf config
file because it was wrong. Here it is :

Also I changed a bit the input config (percpu = false):

In my influxDB console. I created the db telegraf and created the user ‘telegraf’
with the password ‘un_joli_password’ and I granted the all all rights to the user
‘telegraf’ with the followings commands via the influx console :

and

I run my telegraf using this command :

Here’s my influxDB :

Here’s my different config for the datasource (grafana) :

Re-imported the same dashboard for tests purposes :

When I click on import, there’s still no data but at least the config
seems logic to me. After 5 mins and more, I have these errors in the
chrome inspect :

Not sure if it’s related. I also made sure that all the apps that I was running
were autorised by the firewall such as : influxd.exe / influx.exe / telegraf.exe
located in : Control Panel\System and Security\Windows Firewall\Allowed apps
as you send me in your video (thank you)

As always, thank you. I’ll keep looking.

daniellee · March 8, 2018, 3:29pm

Just so you know - in .conf files, the # symbol means the line is commented out and will have no affect. Did you get this working?

Can you use the Query Inspector or the network tab in Chrome Dev Tools to show the queries to InfluxDB and their responses?

The unauthorized error is coming from InfluxDB I think and means the user/password are incorrect.

kawless · March 13, 2018, 3:54pm

Oh, you’re totally right. Thank you for letting me know. I think I did yes. I have a different message popping in the console such as :

Also :

I changed the following in my conf file :

and :

I kept the same config for the datasource and took the same graph
for reference. Here’s the query inspector :

Oh I see, is it better to remove the user and the password from the datasource
and the conf file?

Thank you in advance.

daniellee · March 21, 2018, 12:36pm

Looks like you are getting data into InfluxDB now - in the first screenshot it looks like data is being saved in telegraf.autogen.ping. Can you write a simple query and check that it has data.

I can recommend this plugin for exploring data in InfluxDB (if you want to check which measurements you have in your database): https://grafana.com/plugins/natel-influx-admin-panel

There are a lot of guides to getting started with InfluxDB: