I am aware that grafana can read values from log stash via the elastic search datasource. But can it read values from raw application log files like a springboot application.log. If it can, how to do so ?
Not supported in grafana. However grafana allows custom data source plugins. Don’t know any official plugin that does what you’re asking of, but you could maybe implement one yourself and/or implement a rest api to be used with simple json datasource plugin.
With some research i have found that Telegraf with few input plugins can feed data into influxdb which can feed the results into the grafana dashboard, i havent been successful yet will relay my results
I thought you where referring to querying log files directly from Grafana. Influxdb is probably not what you want to use for logs because of high cardinality problem. Influxdb is a time-series database optimized for storing time-series (timestamp, value and possibly tags). I would suggest you to use elasticsearch if you want to do log aggregation.
When Telegraph has plugins to read apache logs why cant it read raw log files. which too have a pattern. of logging level. It should be possible in my opinion
If you’re interested in parsing log files and storing metrics of these in influxdb it’s supported as you earlier wrote using this input plugin.
I have achieved what i wanted using the
[[inputs.exec]] plugin. this plugin runs a shell script which runs a grep command to count the number of errors in a log file and echo the result, the result is fed it to influxdb with a timestamp of when it occurred. via the [[outputs.influxdb]] plugin
once it is in influxdb i pull the data to grafana and display via a beautiful graph. my need was to count the number of errors in a daily batch file. and alert if there are errors. for legacy reasons i could not feed the logs to logstash which would have made things easier. but still a meet in middle approach helped me do what i needed to
Please share your exec script.