Grafana simplejson dashboard

sagarika · February 21, 2018, 6:27am

Hi,
I am trying to create a dashboard with simplejson data source which we have created.
We have created a simple data source datasource configured with the rest api url which we exposed (in http settings of dashboard we have configured the rest url http://localhost:8060 which we exposed) and grafana is running in 8080 port number. We have exposed the rest GET method and tested the data source which is working fine and could able to see the response. But configuring the same data source in dashboard we couldn’t able to get the metrics and response. It is giving 404 method not found error. Below is the error while creating dashboard.

Possibly unhandled rejection: {“err”:{“data”:null,“status”:-1,“config”:{“method”:“POST”,“transformRequest”:[null],“transformResponse”:[null],“jsonpCallbackParam”:“callback”,“url”:“http://localhost:8060”,“data”:{“target”:""},“headers”:{“Content-Type”:“application/json”,“Accept”:“application/json, text/plain, /”},“retry”:0},“statusText”:"",“xhrStatus”:“error”},“cancelled”:true}

The datasource is configured in such a way that it is giving a response like this below format.

{
“panelId”: 1,
“range”: {
“from”: “2016-10-31T06:33:44.866Z”,
“to”: “2016-10-31T12:33:44.866Z”,
“raw”: {
“from”: “now-6h”,
“to”: “now”
}
},
“rangeRaw”: {
“from”: “now-6h”,
“to”: “now”
},
“interval”: “30s”,
“intervalMs”: 30000,
“output”: [
{ “name”: “hello1”, “classNmae”: “A”, “type”: “timeserie” },
{ “name”: “hello2”, “classNmae”: “B”, “type”: “timeserie” }
],
“format”: “json”,
“maxDataPoints”: 550
}

We are pretty new to grafana so we couldn’t able to figure out what is missed. We implemented rest through GET method but not POST.
Please help us to sort out.

Thanks.

mefraimsson · February 22, 2018, 8:45am

Hi,

Reading about Simple JSON Datasource plugin it says that the request will look like what you’ve posted above and not the response which you indicate. The readme says the following regarding responses from the /query endpoint:

Example timeserie response

[
  {
    "target":"upper_75", // The field being queried for
    "datapoints":[
      [622,1450754160000],  // Metric value as a float , unixtimestamp in milliseconds
      [365,1450754220000]
    ]
  },
  {
    "target":"upper_90",
    "datapoints":[
      [861,1450754160000],
      [767,1450754220000]
    ]
  }
]

If the metric selected is “type”: “table”, an example table response:

[
  {
    "columns":[
      {"text":"Time","type":"time"},
      {"text":"Country","type":"string"},
      {"text":"Number","type":"number"}
    ],
    "rows":[
      [1234567,"SE",123],
      [1234567,"DE",231],
      [1234567,"US",321]
    ],
    "type":"table"
  }
]

Please have a look at https://github.com/bergquist/fake-simple-json-datasource for example implementation in node js. In this example all the endpoints supports all HTTP verbs and not only GET.

Marcus

sagarika · February 23, 2018, 6:42am

Thanks for your reply. Even if i configure the response in such a way what you have posted above i couldn’t able to get the dashboard created with the datasource created.

As per our understanding, the datasource should return a response in such a way that timeserie dashboard should able to display the metrics. The timeserie request should be configured from the datasource.

Please help us to sort out.

Thanks.

mefraimsson · February 23, 2018, 10:48am

Hi,

I’m going to try to explain the steps involved using the SimpleJSON datasource plugin using the fake-simple-json-datasource linked above.

Start fake-simple-json-datasource (nodejs required)

git clone https://github.com/bergquist/fake-simple-json-datasource 
cd fake-simple-json-datasource 
npm install
node index.js

Configure a new Simple JSON datasource in Grafana using the url of your custom built REST API, for example http://localhost:3333 which is default port that the fake-simple-json-datasource are listening on.
- I choose Access=proxy
- Then you save it and it should give you response that it went good

Create a new dashboard
Add a graph panel and go into edit panel
Switch to the metric tab
Select your Simple JSON datasource that you’ve created earlier
- When you’ve selected the Simple JSON datasource the browser are doing a request to fetch available metrics. The Simple JSON datasource plugin will receive the request from Grafana frontend and then issue a request to the url configured, i.e. http://localhost:3333/query. This is how the request/response to/from SimpleJSON datasource looks like in chrome dev tools:

Similar, this is what the request to http://localhost:3333/query from SimpleJSON datasource plugin looks like:

{ timezone: 'browser',
  panelId: 6,
  range:
   { from: '2018-02-23T04:14:50.889Z',
     to: '2018-02-23T10:14:50.889Z',
     raw: { from: 'now-6h', to: 'now' } },
  rangeRaw: { from: 'now-6h', to: 'now' },
  interval: '30s',
  intervalMs: 30000,
  targets: [ { target: '', refId: 'A', type: 'timeserie' } ],
  maxDataPoints: 755,
  scopedVars:
   { __interval: { text: '30s', value: '30s' },
     __interval_ms: { text: 30000, value: 30000 } } }

Select upper_95 metric for example and the following request/response are sent

Similar, this is what the request to http://localhost:3333/query from SimpleJSON datasource plugin looks like:

{ timezone: 'browser',
  panelId: 6,
  range: 
   { from: '2018-02-23T04:31:36.797Z',
     to: '2018-02-23T10:31:36.797Z',
     raw: { from: 'now-6h', to: 'now' } },
  rangeRaw: { from: 'now-6h', to: 'now' },
  interval: '30s',
  intervalMs: 30000,
  targets: [ { target: 'upper_95', refId: 'A', type: 'timeserie' } ],
  maxDataPoints: 755,
  scopedVars: 
   { __interval: { text: '30s', value: '30s' },
     __interval_ms: { text: 30000, value: 30000 } } }

This is my result

Hope this clears up any confusion about how to implement/use the SimpleJSON datasource plugin?

Marcus

mikequentel · May 9, 2018, 9:48pm

Tip/advice if you are rolling your own simplejson service to feed the simplejson datasource plugin…found this out when creating a Python Flask service that consumes documents in MongoDB to expose as a simplejson service (sort of a round-about way to create a MongoDB datasource).

If you are exposing an interface that returns datapoints JSON that can be rendered as a graph panel in Grafana, then:

Ensure that the JSON string does not contain any carriage return or newline characters. For example, remove \r and \n from the JSON output of your service.
If you encounter the error t.dataList.map is not a function then ensure that the JSON emitted from your service has a white character space after each comma (,) and colon (:) …for example:
- BAD (will fail to render a graph): [{"target":"my_metric","datapoints":[[2,1525320000000],[1,1525320000000],[19,1525320000000],[0,1525320000000],[2,1525665600000],[1,1525665600000],[17,1525665600000],[0,1525665600000],[3,1525838400000],[1,1525838400000],[17,1525838400000],[0,1525838400000],[0,1525838400000]]}]
- GOOD: [{"target": "my_metric", "datapoints": [[2, 1525320000000], [1, 1525320000000], [19, 1525320000000], [0, 1525320000000], [2, 1525665600000], [1, 1525665600000], [17, 1525665600000], [0, 1525665600000], [3, 1525838400000], [1, 1525838400000], [17, 1525838400000], [0, 1525838400000], [0, 1525838400000]]}]

ricardogil · July 4, 2018, 8:43pm

Hi @mefraimsson, Im wondering if you know how can I get the scopedVars values when loading the dashboard where the Graph Panel with Json Database exists.
I noticed that when loading for the first time, the current Variable values are not being passed.
If we change the variable dropdown value, then the Json Database will receive that data inside scopedVars object.

Can you help?

mefraimsson · July 5, 2018, 10:10am

Are you using the latest version of simple json datasource? I would recommend to open an issue in the simple json repo.

ricardogil · July 5, 2018, 12:21pm

Yes, Im using id: grafana-simple-json-datasource version: 1.4.0 and Grafana 5.1.0

The reason I did not open issue to the simple json repo is because I think it is an issue at Grafana side. Let me explain why:
1-I took a look at the json repo code and Im sure the query(options) is receiving “raw” options object from Grafana , so it seems that if the scopedVars are present or not depends on what Grafana is sending when call query(options)
2- If I use a variable of type Query and load my dashboard from the first time, this is the value of the request body (see that the variable testQuery is populated)
dashboardId:4
interval:“5m”
intervalMs:300000
maxDataPoints:1867
panelId:10
range:Object {from: “2018-06-28T12:14:51.917Z”, to: “2018-07-05T12:14:51.917Z”, raw: Object}
rangeRaw:Object {from: “now-7d”, to: “now”}
scopedVars:Object {testQuery: Object, __interval: Object, __interval_ms: Object}
__interval:Object {text: “5m”, value: “5m”}
__interval_ms:Object {text: 300000, value: 300000}
testQuery:Object {text: “test”, value: “test”, selected: true}
selected:true
text:“testt”
value:“test”
targets:Array[1] [Object]
timezone:"browser

3-But (here I think is the bug), if I change in my panel and add to the “For each value of” a Custom variable:

when I load the dashboard for the first time, this is the request body content:

So my question is: Why when the variable is type QUERY, the value goes to the query(options object) and when it is CUSTOM variable it only goes if I change the variable dropdown value?

mefraimsson · July 5, 2018, 3:20pm

Created an example dashboard which repeat panels by custom variable. Seems to work for me and therefore I’m not so sure what you report as a Grafana problem.

rakesh111 · December 4, 2018, 12:11am

My python(Flask) code for simple json plugin .

from flask import  Flask
import cx_Oracle

app = Flask(__name__)

@app.route('/')
def healthtest():
return "Good" 

@app.route('/search', methods=['POST'])
req = req.get_json():
def searching():
return searching

#“I am not understanding how to write the /search, /query, targets, “range from” and “range to” please help me out”

@app.route('/query',methods=['GET'])
if  target == "Food":
conn = cx_Oracle.connect(user='',password='',dsn=dsn_tns)
c.execute('''SELECT APPLE, BANANA, CARROT FROM VEGETABLES''')
for row in c:
print (row)
conn.close()
else :
print "No DATA"

app.run(host=0.0.0.0, port=0000, debug=True)

please help me with how to parse the json and how to write the search and the query , targets, range from and range to. Very less documentation on this.
My query paramter is “Food”
My columns are “Vegetables” and “fruits”

Please help me out how to parse my json in grafana table in python(Flask)

mefraimsson · December 4, 2018, 8:13am

I don’t think anyone will write the code for you, but should be fairly trivial for you to write in python given the node js example.

rakesh111 · December 4, 2018, 11:57pm

from flask import  Flask, jsonify, request
import cx_Oracle

app = Flask(__name__)
app.debug = True

conn = cx_Oracle.connect(user='',password='',dsn=dsn_tns)
c.execute('''SELECT APPLE, BANANA, CARROT FROM VEGETABLES''')
for row in c:
print (row)  # my json output from database will be here in "row"

def mydata(row):
     return (row)


@app.route('/')
def healthtest():
return "Good" 

 @app.route('/search', methods=['POST'])
def searching():
req = req.get_json():
return jsonify(["targets"])

@app.route('/query',methods=['POST'])
def query():
req = request.get_json()
data = [
{                      "target": row["targets"][0]["target"],
                        "datapoints":[
                                   [2000, row(req["range"]["from"]],
                                   [2000, row(req["range"]["from"]]
            ]
                 }      
   ]

    return jsonify(data)

if __name__ = "__main__":
 app.run()

my “/” is working but my “/search” and “/query” is not working. I want to just print the database data in grafana table with two columns. fruits, vegetables with the parameter with the method name “FOOD” how would I write to a method to declare please help me out, guys. please correct my mistakes “/search” “/query” .it would be greatly appreciated. I am very new to python and grafana .

rakesh111 · December 7, 2018, 6:06am

@mefraimsson How to convert grafana expected way of columns to my keys and my values to grafana way of rows in python dynamically. is there any lambda or any script in python for this very fewer resources in online and not clear ?? please help me out guys I tried in many different ways couldn’t figure it out

   myjson = [{"one":"1","two":"2"},
                  {"one":"3","two":"4"}]


   ** Grafana expected json format structure for converting json to table format**
               [
           {
       "columns":[
        {"text":"Time","type":"time"},
      {"text":"Country","type":"string"},
         {"text":"Number","type":"number"}
      ],
      "rows":[
        [1234567,"SE",123],
      [1234567,"DE",231],
      [1234567,"US",321]
    ] ,
     "type":"table"
     }
    ]

arvindms · January 6, 2019, 8:49pm

How is the Tag Keys API call triggered from the dashboard? Created a variable of type adhoc filter, but that did not seem to call this endpoint.

thefamouseric · January 7, 2019, 1:39pm

should the $custom variable be available in the actual JSON query

sanfx · November 9, 2019, 5:02pm

tldr;
What is the meaning of doing this ?
Does it mean when you http://serverAddress/search?targets

rohollahehsani · November 10, 2020, 12:31pm

Hi @mefraimsson,

I have a question around this topic!
My REST API server is secured through token-based Authentication! meaning that I have to pass a token along with the API request in order to retrieve the data!

here is an example of a request:
curl -H "Auth: <–token–> " "http://–link_url–/*/api/ "

Is there any solution regarding this problem!

Thank you in advance!

Topic		Replies	Views
Grafana API create dashboard API with Python Developers & API api	6	11050	June 29, 2022
Help with dashboard provisioning Grafana provisioning	4	4682	April 16, 2018
Bad Gateway Errors Configuration	33	105367	November 5, 2021
Display JSON from Microsoft Graph HTTP Request to Grafana Dashboard Dashboards infinity-datasource	4	38	September 4, 2024
Template variable service failed Failed to fetch Dashboards templating	13	3329	October 13, 2023

Grafana simplejson dashboard

Related Topics