Grafana simplejson dashboard


#1

Hi,
I am trying to create a dashboard with simplejson data source which we have created.
We have created a simple data source datasource configured with the rest api url which we exposed (in http settings of dashboard we have configured the rest url http://localhost:8060 which we exposed) and grafana is running in 8080 port number. We have exposed the rest GET method and tested the data source which is working fine and could able to see the response. But configuring the same data source in dashboard we couldn’t able to get the metrics and response. It is giving 404 method not found error. Below is the error while creating dashboard.

Possibly unhandled rejection: {“err”:{“data”:null,“status”:-1,“config”:{“method”:“POST”,“transformRequest”:[null],“transformResponse”:[null],“jsonpCallbackParam”:“callback”,“url”:“http://localhost:8060”,“data”:{“target”:""},“headers”:{“Content-Type”:“application/json”,“Accept”:“application/json, text/plain, /”},“retry”:0},“statusText”:"",“xhrStatus”:“error”},“cancelled”:true}

The datasource is configured in such a way that it is giving a response like this below format.

{
“panelId”: 1,
“range”: {
“from”: “2016-10-31T06:33:44.866Z”,
“to”: “2016-10-31T12:33:44.866Z”,
“raw”: {
“from”: “now-6h”,
“to”: “now”
}
},
“rangeRaw”: {
“from”: “now-6h”,
“to”: “now”
},
“interval”: “30s”,
“intervalMs”: 30000,
“output”: [
{ “name”: “hello1”, “classNmae”: “A”, “type”: “timeserie” },
{ “name”: “hello2”, “classNmae”: “B”, “type”: “timeserie” }
],
“format”: “json”,
“maxDataPoints”: 550
}

We are pretty new to grafana so we couldn’t able to figure out what is missed. We implemented rest through GET method but not POST.
Please help us to sort out.

Thanks.


Grafana dashboard with simplejson
#2

Hi,

Reading about Simple JSON Datasource plugin it says that the request will look like what you’ve posted above and not the response which you indicate. The readme says the following regarding responses from the /query endpoint:

Example timeserie response

[
  {
    "target":"upper_75", // The field being queried for
    "datapoints":[
      [622,1450754160000],  // Metric value as a float , unixtimestamp in milliseconds
      [365,1450754220000]
    ]
  },
  {
    "target":"upper_90",
    "datapoints":[
      [861,1450754160000],
      [767,1450754220000]
    ]
  }
]

If the metric selected is “type”: “table”, an example table response:

[
  {
    "columns":[
      {"text":"Time","type":"time"},
      {"text":"Country","type":"string"},
      {"text":"Number","type":"number"}
    ],
    "rows":[
      [1234567,"SE",123],
      [1234567,"DE",231],
      [1234567,"US",321]
    ],
    "type":"table"
  }
]

Please have a look at https://github.com/bergquist/fake-simple-json-datasource for example implementation in node js. In this example all the endpoints supports all HTTP verbs and not only GET.

Marcus


#3

Thanks for your reply. Even if i configure the response in such a way what you have posted above i couldn’t able to get the dashboard created with the datasource created.

As per our understanding, the datasource should return a response in such a way that timeserie dashboard should able to display the metrics. The timeserie request should be configured from the datasource.

Please help us to sort out.

Thanks.


#4

Hi,

I’m going to try to explain the steps involved using the SimpleJSON datasource plugin using the fake-simple-json-datasource linked above.

  1. Start fake-simple-json-datasource (nodejs required)
git clone https://github.com/bergquist/fake-simple-json-datasource 
cd fake-simple-json-datasource 
npm install
node index.js
  1. Configure a new Simple JSON datasource in Grafana using the url of your custom built REST API, for example http://localhost:3333 which is default port that the fake-simple-json-datasource are listening on.
    • I choose Access=proxy
    • Then you save it and it should give you response that it went good

  1. Create a new dashboard

  2. Add a graph panel and go into edit panel

  3. Switch to the metric tab

  4. Select your Simple JSON datasource that you’ve created earlier

    • When you’ve selected the Simple JSON datasource the browser are doing a request to fetch available metrics. The Simple JSON datasource plugin will receive the request from Grafana frontend and then issue a request to the url configured, i.e. http://localhost:3333/query. This is how the request/response to/from SimpleJSON datasource looks like in chrome dev tools:

Similar, this is what the request to http://localhost:3333/query from SimpleJSON datasource plugin looks like:

{ timezone: 'browser',
  panelId: 6,
  range:
   { from: '2018-02-23T04:14:50.889Z',
     to: '2018-02-23T10:14:50.889Z',
     raw: { from: 'now-6h', to: 'now' } },
  rangeRaw: { from: 'now-6h', to: 'now' },
  interval: '30s',
  intervalMs: 30000,
  targets: [ { target: '', refId: 'A', type: 'timeserie' } ],
  maxDataPoints: 755,
  scopedVars:
   { __interval: { text: '30s', value: '30s' },
     __interval_ms: { text: 30000, value: 30000 } } }
  1. Select upper_95 metric for example and the following request/response are sent

Similar, this is what the request to http://localhost:3333/query from SimpleJSON datasource plugin looks like:

{ timezone: 'browser',
  panelId: 6,
  range: 
   { from: '2018-02-23T04:31:36.797Z',
     to: '2018-02-23T10:31:36.797Z',
     raw: { from: 'now-6h', to: 'now' } },
  rangeRaw: { from: 'now-6h', to: 'now' },
  interval: '30s',
  intervalMs: 30000,
  targets: [ { target: 'upper_95', refId: 'A', type: 'timeserie' } ],
  maxDataPoints: 755,
  scopedVars: 
   { __interval: { text: '30s', value: '30s' },
     __interval_ms: { text: 30000, value: 30000 } } }

  1. This is my result

Hope this clears up any confusion about how to implement/use the SimpleJSON datasource plugin?

Marcus


How to import REST API Data
JSON Datasource
SimpleJSON can modify format of serie?
Event visualisation on timeseries graph (sunset / sunrise over time)
Unable to get response from json request
HELP..URGENT about grafana
How to associate LogicMonitor Graph data with the grafana SampleJson plugin?
#5

Tip/advice if you are rolling your own simplejson service to feed the simplejson datasource plugin…found this out when creating a Python Flask service that consumes documents in MongoDB to expose as a simplejson service (sort of a round-about way to create a MongoDB datasource).

If you are exposing an interface that returns datapoints JSON that can be rendered as a graph panel in Grafana, then:

  • Ensure that the JSON string does not contain any carriage return or newline characters. For example, remove \r and \n from the JSON output of your service.

  • If you encounter the error t.dataList.map is not a function then ensure that the JSON emitted from your service has a white character space after each comma (,) and colon (:) …for example:

    • BAD (will fail to render a graph): [{"target":"my_metric","datapoints":[[2,1525320000000],[1,1525320000000],[19,1525320000000],[0,1525320000000],[2,1525665600000],[1,1525665600000],[17,1525665600000],[0,1525665600000],[3,1525838400000],[1,1525838400000],[17,1525838400000],[0,1525838400000],[0,1525838400000]]}]

    • GOOD: [{"target": "my_metric", "datapoints": [[2, 1525320000000], [1, 1525320000000], [19, 1525320000000], [0, 1525320000000], [2, 1525665600000], [1, 1525665600000], [17, 1525665600000], [0, 1525665600000], [3, 1525838400000], [1, 1525838400000], [17, 1525838400000], [0, 1525838400000], [0, 1525838400000]]}]


#6

Hi @mefraimsson, Im wondering if you know how can I get the scopedVars values when loading the dashboard where the Graph Panel with Json Database exists.
I noticed that when loading for the first time, the current Variable values are not being passed.
If we change the variable dropdown value, then the Json Database will receive that data inside scopedVars object.

Can you help?


#7

Are you using the latest version of simple json datasource? I would recommend to open an issue in the simple json repo.


#8

Yes, Im using id: grafana-simple-json-datasource version: 1.4.0 and Grafana 5.1.0

The reason I did not open issue to the simple json repo is because I think it is an issue at Grafana side. Let me explain why:
1-I took a look at the json repo code and Im sure the query(options) is receiving “raw” options object from Grafana , so it seems that if the scopedVars are present or not depends on what Grafana is sending when call query(options)
2- If I use a variable of type Query and load my dashboard from the first time, this is the value of the request body (see that the variable testQuery is populated)
dashboardId:4
interval:“5m”
intervalMs:300000
maxDataPoints:1867
panelId:10
range:Object {from: “2018-06-28T12:14:51.917Z”, to: “2018-07-05T12:14:51.917Z”, raw: Object}
rangeRaw:Object {from: “now-7d”, to: “now”}
scopedVars:Object {testQuery: Object, __interval: Object, __interval_ms: Object}
__interval:Object {text: “5m”, value: “5m”}
__interval_ms:Object {text: 300000, value: 300000}
testQuery:Object {text: “test”, value: “test”, selected: true}
selected:true
text:“testt”
value:“test”
targets:Array[1] [Object]
timezone:"browser

3-But (here I think is the bug), if I change in my panel and add to the “For each value of” a Custom variable:
image
when I load the dashboard for the first time, this is the request body content:

So my question is: Why when the variable is type QUERY, the value goes to the query(options object) and when it is CUSTOM variable it only goes if I change the variable dropdown value?


#9

Created an example dashboard which repeat panels by custom variable. Seems to work for me and therefore I’m not so sure what you report as a Grafana problem.


#10

My python(Flask) code for simple json plugin .

from flask import  Flask
import cx_Oracle

app = Flask(__name__)

@app.route('/')
def healthtest():
return "Good" 

@app.route('/search', methods=['POST'])
req = req.get_json():
def searching():
return searching

#“I am not understanding how to write the /search, /query, targets, “range from” and “range to” please help me out”

@app.route('/query',methods=['GET'])
if  target == "Food":
conn = cx_Oracle.connect(user='',password='',dsn=dsn_tns)
c.execute('''SELECT APPLE, BANANA, CARROT FROM VEGETABLES''')
for row in c:
print (row)
conn.close()
else :
print "No DATA"

app.run(host=0.0.0.0, port=0000, debug=True)

please help me with how to parse the json and how to write the search and the query , targets, range from and range to. Very less documentation on this.
My query paramter is “Food”
My columns are “Vegetables” and “fruits”

Please help me out how to parse my json in grafana table in python(Flask)


#11

I don’t think anyone will write the code for you, but should be fairly trivial for you to write in python given the node js example.


#12
from flask import  Flask, jsonify, request
import cx_Oracle

app = Flask(__name__)
app.debug = True

conn = cx_Oracle.connect(user='',password='',dsn=dsn_tns)
c.execute('''SELECT APPLE, BANANA, CARROT FROM VEGETABLES''')
for row in c:
print (row)  # my json output from database will be here in "row"

def mydata(row):
     return (row)


@app.route('/')
def healthtest():
return "Good" 

 @app.route('/search', methods=['POST'])
def searching():
req = req.get_json():
return jsonify(["targets"])

@app.route('/query',methods=['POST'])
def query():
req = request.get_json()
data = [
{                      "target": row["targets"][0]["target"],
                        "datapoints":[
                                   [2000, row(req["range"]["from"]],
                                   [2000, row(req["range"]["from"]]
            ]
                 }      
   ]

    return jsonify(data)

if __name__ = "__main__":
 app.run()

my “/” is working but my “/search” and “/query” is not working. I want to just print the database data in grafana table with two columns. fruits, vegetables with the parameter with the method name “FOOD” how would I write to a method to declare please help me out, guys. please correct my mistakes “/search” “/query” .it would be greatly appreciated. I am very new to python and grafana .


#13

@mefraimsson How to convert grafana expected way of columns to my keys and my values to grafana way of rows in python dynamically. is there any lambda or any script in python for this very fewer resources in online and not clear ?? please help me out guys I tried in many different ways couldn’t figure it out

   myjson = [{"one":"1","two":"2"},
                  {"one":"3","two":"4"}]


   ** Grafana expected json format structure for converting json to table format**
               [
           {
       "columns":[
        {"text":"Time","type":"time"},
      {"text":"Country","type":"string"},
         {"text":"Number","type":"number"}
      ],
      "rows":[
        [1234567,"SE",123],
      [1234567,"DE",231],
      [1234567,"US",321]
    ] ,
     "type":"table"
     }
    ]