Grafana Cloudwatch Plugin

Grafana Cloudwatch
1

  • What Grafana version and what operating system are you using?

    Grafana™ software 8.2 on Ubuntu 22.04 LTS

  • What are you trying to achieve?

    Connection with Cloudwatch Logs

  • How are you trying to achieve it?

    Cloudwatch Plugin

  • What happened?

    Hi. I am trying to connect to Cloudwatch Logs via the Plugin. It auctually works, however I just want to get out the Logs of Cloudwatch and not the metrics and it gives me the following error which is below. The Problem is that I have to specify only one endpoint and as there are two different endpoints for the Logs and the Metrics in Cloudwatch I get the error for the metrics as the plugin is trying to get them out of the logs endpoint. This seems to be a bug for me but maybe there is a simple way to fix this?

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.

    CloudWatch metrics query failed: SerializationError: failed to unmarshal error message status code: 404, request id: d36b0a23-8741-48f6-a1ea-325ca85fb838 caused by: UnmarshalError: failed to unmarshal error message 00000000 3c 55 6e 6b 6e 6f 77 6e 4f 70 65 72 61 74 69 6f |<UnknownOperatio| 00000010 6e 45 78 63 65 70 74 69 6f 6e 2f 3e 0a |nException/>.| caused by: unknown error response tag, {{ UnknownOperationException} } 2. Successfully queried the CloudWatch logs API.

2

That is an error for CloudWatch metric, but you didn’t set a proper permissions for that so error for metrics is expected. I would ignore it. 2. Successfully queried the CloudWatch logs API., so you are good to work with logs.

3

Hi,

thank you for the answer. This should not be a permission problem as I have given full permission to my role. I think, as already mentioned the problem is that I specified the Cloudwatch Logs endpoint and is trying to get the metrics out of this endpoint. This gives an error because the metrics cannot be retrieved from the Cloudwatch Logs endpoint and I cannot specify both endpoints.

4

I would create 2 datasources in this case with different endpoints - one for metrics and second one for logs.

1 Like
5

Hi @jacopozurbuch!

Created the account only to document the solution to this error, was facing the same one.

I’m running Grafana 9.2.5 in ECS, and while trying to add CloudWatch as a DataSource, was getting the same deserialization error. Checking the CloudWatch logs, it was some mention of regions, which led me astray, as the documented permissions here: AWS CloudWatch | Grafana documentation where in place…

In the DataSource menu, there is a mention of XRay permissions, but in the docs it doesn’t look like they are mandatory… well, looks like the XRay permissions are used somehow (haven’t dug into the plugin source code though).

Added the AWSXrayReadOnlyAccess AWS managed policy to the ECS task role, and the CloudWatch DataSource works. Can make queries, example dashboards loads, etc.

Hope that this saves time to the next person :slight_smile:

TL;DR: Try with XRay read permissions to the user/role