Synthetic monitoring on local grafana instance

sureshpalanisamy · January 31, 2022, 6:02pm

Hello,

i have been installed the grafana synthetic monitoring on my grafana local instance. i could able to see the synthetic monitoring plugin on my local grafana instance. when i am trying to configure the plugin it was keep on loading. i could not find any progress. could you please help me out.

Regards,
Suresh

melori.arellano · February 3, 2022, 12:45am

Hi @sureshpalanisamy, welcome to the community!

Synthetic monitoring runs on your cloud instance, but you can set up the plugin on your local grafana to view it locally. It will still need to be configured to connect to your cloud instance to get the synthetic monitoring data.

Are you following the instructions here for configuring the connection with your cloud instance?

sureshpalanisamy · February 16, 2022, 5:44pm

Hi @melori.arellano , Thanks for the update.

No i didn’t create an yaml file. i have just downloaded the plugin and extracted on my grafana local instance. do i need to create the provisioning file under plugin directory?

Regards,
Suresh

sureshpalanisamy · February 17, 2022, 12:08pm

@melori.arellano I have created both file under plugin directory of my grafana local instance. still i could see it is keep on loading screen.

Regards,
Suresh

melori.arellano · February 17, 2022, 4:49pm

@sureshpalanisamy, I recommend enabling debug logging and restarting grafana. You will be able to see if there are any errors in the plugin configuration.

Enable debug logs

sureshpalanisamy · March 9, 2022, 11:12am

@melori.arellano Hi sorry for the delay. i had some health issue and not able to connect . i have enabled the debug log and i did’nt found any error to troubleshoot. i have created my both files under plugins directory. still no luck.

Regards,
Suresh

melori.arellano · March 10, 2022, 12:01am

@sureshpalanisamy, I hope you’re recovering well. I re-labeled this for grafana cloud.

What path is set in grafana.ini for provisioning, verify that it matches where your file is. The line should look like this:

# folder that contains provisioning config files that grafana will apply
# on startup and while running.

provisioning = conf/provisioning

mem · March 10, 2022, 1:07pm

Hi Suresh,

you need to provision the Synthetic Monitoring plugin and the Grafana Cloud datasources. While you can install the plugin in a local Grafana instance, you need a hosted Grafana stack where you can store the data. At the moment there’s no support for storing the data in self-hosted instances of Prometheus and Loki.

If that’s fine, then you need two files:

/etc/grafana/provisioning/plugins/sm.yaml (name is not important, location is, please check the local Grafana configuration for the correct location)

/etc/grafana/provisioning/datasources/grafana_cloud.yaml (same)

For the datasources it’s something like:

apiVersion: 1

datasources:
  - name: grafanacloud-sm-logs
    type: loki
    access: proxy
    url: ...
    basicAuth: true
    basicAuthUser: ...
    jsonData:
      maxLines: 1000
    secureJsonData:
      basicAuthPassword: ...
    version: 1

  - name: grafanacloud-sm-metrics
    type: prometheus
    access: proxy
    url: ...
    basicAuth: true
    basicAuthUser: ...
    jsonData:
      timeInterval: 1s
    secureJsonData:
      basicAuthPassword: ...
    version: 1

For the plugin:

apiVersion: 1

apps:
  - type: grafana-synthetic-monitoring-app
    name: grafana-synthetic-monitoring-app
    disabled: false
    jsonData:
      apiHost: ...
      stackId: ...
      logs:
        grafanaName: grafanacloud-sm-logs
        hostedId: ...
      metrics:
        grafanaName: grafanacloud-sm-metrics
        hostedId: ...
    secureJsonData:
      publisherToken: ...

The names of the datasources do not matter, but they must be consistent.

The tokens for the datasources are viewer tokens (or editor, for some use cases).

The token for the plugin is a publisher token.

There’s a legacy set up path that requires an admin token, but I would recommend that you try this first.

sureshpalanisamy · March 14, 2022, 3:53pm

@mem @melori.arellano Thanks a lot for your time.

Let me check the above option and back to you.

Regards,
Suresh

sureshpalanisamy · March 14, 2022, 4:49pm

@melori.arellano Please find my location file

Directory where grafana will automatically scan and look for plugins

;plugins = /var/lib/grafana/plugins

folder that contains provisioning config files that grafana will apply on startup and while running.

;provisioning = conf/provisioning

@mem i can go with prometheus and loki for storing

can you help me should i create both yaml under /var/lib/grafana/plugins.

is it possible if we can have screen share session. because i have tried for many times but still no luck.

melori.arellano · March 14, 2022, 8:37pm

@sureshpalanisamy I’m reading the docs on github for setting this up and I think we can improve them to make this easier.

Here is what you’ll want to do:

Check that the plugin grafana-synthetic-monitoring-app is installed in your plugins directory /var/lib/grafana/plugins. If it is not, Install the synthetic monitoring plugin on your local instance of grafana into the plugin directory
Configure your local instance of grafana to connect to your cloud prometheus and loki instances so you can view their data in your local instance of grafana.

Create an API key with Viewer permissions in the Grafana Cloud portal:

Screen Shot 2022-03-14 at 2.32.09 PM841×467 38.1 KB
Log in to your grafana cloud instance and refer to the grafanacloud-XXXX-prom and grafanacloud-XXXX-logs datasources to get the URL and user name that you will use to configure the datasources locally

Screen Shot 2022-03-14 at 2.37.19 PM638×874 55.2 KB
In your local instance of grafana under Configuration > Datasources > Add Datasource. Create datasource configurations for loki and prometheus to match the ones that exist in grafana cloud. Use your API Viewer key as the password for both.

Note: The user is the hostedId that you will need to configure the provisioning file. Keep this browser tab open to reference in step 3

Configure the synthetic-monitoring-app using a provisioning file in your conf/provisioning directory.

Create an API key with the MetricsPublisher permission
Verify that your plugin provisioning file looks like the one @mem shared and the example here.

apiVersion: 1

apps:
  - type: grafana-synthetic-monitoring-app
    name: grafana-synthetic-monitoring-app
    disabled: false
    jsonData:
      apiHost: ...
      stackId: ...
      logs:
        grafanaName: grafanacloud-sm-logs
        hostedId: ...
      metrics:
        grafanaName: grafanacloud-sm-metrics
        hostedId: ...
    secureJsonData:
      publisherToken: ...

stackId is in the URL when you navigate to your stack in grafana.com:

hostedId is the user name that you saw in your loki and prometheus datasources in step 2. They are specific to your prometheus (metrics) and loki (logs) instances.
apiHost URLs are region specific and listed here
publisherToken is the API key with the MetricsPublisher permission
grafanaName is the name of local logs and metrics datasources you created

Restart grafana

sureshpalanisamy · March 15, 2022, 10:03am

@melori.arellano i have done all the steps but still the app is not loading on the plugin.

Regards,
Suresh

melori.arellano · March 15, 2022, 4:29pm

@sureshpalanisamy with debug logging on you should see a line like this when the plugin loads:

DBUG[03-15|10:19:09] Loading plugin                           logger=plugin.loader path=/usr/local/var/lib/grafana/plugins/grafana-synthetic-monitoring-app/datasource/plugin.json
DBUG[03-15|10:19:09] Loading plugin                           logger=plugin.loader path=/usr/local/var/lib/grafana/plugins/grafana-synthetic-monitoring-app/plugin.json

If there is a configuration problem, then you might see more errors further down or in the browser developer tools. Are you able to see the synthetic monitoring icon
in your local grafana to indicate that the plugin is installed?

You should also see a line where grafana finds your provisioning file:

t=2022-03-15T10:28:58-0600 lvl=dbug msg="Parsing plugin provisioning file" logger=provisioning.plugins path=/Users/meloriarellano/Downloads/grafana-8.3.3/conf/provisioning/plugins file.Name=sm.yaml

sureshpalanisamy · March 16, 2022, 12:48pm

@melori.arellano hi i have not installed the plugin using grafana cli. i have download the zip file and extracted the on the /var/lib/grafana/plugins directory and on the logs i don’t have any error and plugin was successfully. but once i click on the UI it is keep on loading

sureshpalanisamy · March 31, 2022, 1:44pm

@melori.arellano Hi Hope you are doing good. I have done the steps. now i am getting page not found. when i am clicking on the synthetic monitoring app. Could you please help on this.

melori.arellano · May 20, 2022, 12:01am

@sureshpalanisamy I’m not sure why you’d see that error, are you able to load the page properly when you’re using synthetic monitoring in your grafana cloud instance? If there is an issue with it in your instance, you can open a support ticket with grafana cloud support.

lalitprasanth · October 3, 2022, 6:55am

Hi @melori.arellano , I have followed all the steps as mentioned above and in the debug logs I could see the following

But in grafana dashboard when i am trying to configure the API key That was generated from grafana cloud using MetricsPublisher role, I am getting the below error. Please help me on this.

This is the error i am seeing in logs-
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-10-03T07:16:33.85523627Z level=info msg=“Request Completed” method=POST path=/api/plugin-proxy/grafana-synthetic-monitoring-app/init status=404 remote_addr=49.205.209.219 time_ms=3 duration=3.068941ms size=24 referer=https://grafana.fasal.co/a/grafana-synthetic-monitoring-app/unprovisioned traceID=00000000000000000000000000000000

simonsomerville · January 20, 2023, 2:12pm

Hi everyone,

i have a problem using the SyntheticMonitoring.
I Configured everything like @mem @melori.arellano Described.
( Thanks for that, good Tutorial )

Now iam stuck on local site.
I can see the SyntheticMonitoring that was installed via CLI but i can not press the Initialize the plugin Button, its greyed and not possible to Click.

Any Solutions for that ?

Iam on Local Grafana 8.3.4

Thanks

vladimirb1 · January 7, 2024, 6:24pm

Hi,

I’m trying to install Synthetic Monitoring on local Grafana instance. I’m stuck at “Provisioning missing or invalid. Provisioning is required for Synthetic Monitoring.” Can’t figure out how to setup provisioning…

Topic		Replies	Views
Synthetic Monitoring Plugin Initialization Error Grafana Cloud plugins , synthetic-monitoring	3	1087	January 13, 2023
Error with local synthetic monitoring Grafana Cloud	2	1272	February 15, 2022
[ERROR] Cannot connect grafana cloud with local grafana instance Configuration plugins , synthetic-monitoring	5	431	November 21, 2024
Cannot initialize Synthetic Monitoring Plugin Grafana Cloud synthetic-monitoring	3	942	August 2, 2022
Grafana Synthetic Monitoring Plugin return `Something went wrong` Configuration plugins , synthetic-monitoring	0	13	August 15, 2024

Synthetic monitoring on local grafana instance

Directory where grafana will automatically scan and look for plugins

folder that contains provisioning config files that grafana will apply on startup and while running.

Related topics