So I inherited a grafana environment due to some layoffs and as such after some maintenance last week our grafana environment is throwing a 503 error which seems to be due to a bad datasource(s). Since I can’t login to the interface I’d like to know if its possible to disable one or more datasources from the commandline temporarily so I can which datasource has the issue? I’ve looked in the logs, but they are NOT very helpful and are just showing:
=========
t=2019-03-09T17:18:20-0600 lvl=eror msg=“Alert Rule Result Error” logger=alerting.resultHandler ruleId=10 name=“MW Search Manager Response Times alert” error=“tsdb.HandleRequest() error Request failed status: 500 INTERNAL SERVER ERROR” changing state to=keep_state
t=2019-03-09T17:18:20-0600 lvl=dbug msg=“Job Execution completed” logger=alerting.engine timeMs=159.322 alertId=10 name=“MW Search Manager Response Times alert” firing=true
t=2019-03-09T17:18:20-0600 lvl=info msg=“Request failed” logger=tsdb.graphite status=“500 INTERNAL SERVER ERROR” body="<body style=“background-color: #666666; color: black;”>\n\n\n
Graphite encountered an unexpected error while handling your request.
\nPlease contact your site administrator if the problem persists.
\n\n\n<div style=“width: 50%; text-align: center; font-family: monospace; background-color: black; font-weight: bold; color: #ff4422;”>\n\n\n\n<div style=“width: 70%; text-align: left; background-color: black; color: #44ff22; border: thin solid gray;”>\n
\nTraceback (most recent call last):\n File “/usr/local/rnt/lib/python2.7/site-packages/django/core/handlers/base.py”, line 111, in get_response\n response = wrapped_callback(request, *callback_args, **callback_kwargs)\n File “/usr/local/rnt/webapp/graphite/render/views.py”, line 112, in renderView\n seriesList = evaluateTarget(requestContext, target)\n File “/usr/local/rnt/webapp/graphite/render/evaluator.py”, line 8, in evaluateTarget\n result = evaluateTokens(requestContext, tokens)\n File “/usr/local/rnt/webapp/graphite/render/evaluator.py”, line 29, in evaluateTokens\n return evaluateTokens(requestContext, tokens.expression, replacements)\n File “/usr/local/rnt/webapp/graphite/render/evaluator.py”, line 45, in evaluateTokens\n return fetchData(requestContext, expression)\n File “/usr/local/rnt/webapp/graphite/render/datalib.py”, line 144, in fetchData\n raise e\nZeroDivisionError: integer division or modulo by zero\n\n\n\n\n\n"
I’d like to get this back up and running. I’m seeing the following in
…/grafana/public/app/plugins/datasource directory
elasticsearch grafana grafana-live graphite mixed opentsdb
Anything would be helpful… thanks!