Does log truncation cause a synthetics test to fail?

Grafana Cloud
1

I signed up to Grafana Cloud last week to test the synthetics monitoring solution. If time permits I’d like to contrast the tech here with offerings from Pingdom, New Relic, and BetterStack. The engineering teams where I am would like to be better at prod monitoring, so I’m digging to a bit of real-world testing.

I have set up a monitor here https://xxx.grafana.net/a/grafana-synthetic-monitoring-app/checks (subdomain removed). I’ve written a bit of JavaScript and it sort of works, but I’ve hit into a problem.

First, let me set out a consideration that I think we will have regardless of provider. We’re a travel company offering a number of destinations in a number of locales; roughly 80 each in two locales and 40 in a third. Each of them have their own page, and the availability of each depends on all sorts of complex publishing rules. Now it would be silly to have 200 separate monitors, not least because maintaining that in any tool would be a nightmare. So I am minded to have three monitors here, one per locale.

This is a bit of a trade-off, but three checks seems the right balance to me. If we get a failure we can see that quickly on the (very good) dashboard, and then we can dig into logs to see which one(s) failed.

Now, if I go to edit the monitor (https://xxx.grafana.net/a/grafana-synthetic-monitoring-app/checks/1491/edit) and go to the Alerting section, I can click “Test”. The response headers on some of our (Vercel) endpoints are very chunky, and they seem to result in an error, which is:

Metrics output truncated at 102400 bytes

From my research, this should not stop the test, but the Test feature reports Failed, even though I don’t think it actually has. However it rather spoils the workflow; we can’t publish a test that has not had a successful run, and more than ~30 HTTP calls seems to result in an unavoidable error, which really ought to only be a warning.

So, I have some questions:

  • Is there a way to turn off verbose logging on fetches? I do my own console.log() and console.error() in the JavaScript, which is much more succinct.
  • Is there a way to test a script after editing it that ignores this warning? For example, could we run the code locally?
  • Would any folks working on this product be inclined to regard this case as a bug? I don’t think the test loop is aborted by the truncation report, and thus I think the UI should report a Success state, not a Failure.

Related: Synthetic Monitoring using K6 Script - metrics and log output truncated