Query to identify slowest endpoints for an API (in terms of target_processing_time)

Grafana Dashboards
,
1
  • What Grafana version and what operating system are you using?

Grafana v10.0.3-cloud.4.aed62623 (d9eea1cb64)

  • What are you trying to achieve?

The datasource is an ALB access logstream, where events are access requests for an aiohttp application that include strings of the form:

"GET https://<app name.org domain>:443/api/<endpoint> HTTP/2.0"

where endpoint is a string which can satisfy one of the following forms/schemes:

A
A/B
A/B/<param>
A/B/C
A/B/C/<param>
A/B/C/D
A/B/C/D/<param>

where A are alphanumeric strings, possibly including - and/or _, and <param> is a parameter or parameters, which are either integers or dates in the YYYY-MM-DD format.

I need to identify the endpoints uniquely up to the name parts (excluding the parameter parts), calculate the top 10 or 20 slowest endpoints, in descending order of target processing time.

  • How are you trying to achieve it?
    My current query is only trying to identify the endpoints up to the first part, A, and treating the rest as a remainder string (R). Here’s the full query.
avg by (A, R) (
    sum_over_time (
        {__aws_s3_lb="<ALB instance name>"}
        |= `/api`
        | pattern `<type> <timestamp> <elb> <client_ip> <target_ip> <request_processing_time> <target_processing_time> <response_processing_time> <elb_status_code> <target_status_code> <received_bytes> <sent_bytes> "<method> <_>/api/<endpoint> <_>"`
        | elb_status_code = 200
        | regexp `.*/api/(?P<A>[\w\-_]*)(?P<R>/.*)?`
        | target_processing_time > 0
        | unwrap target_processing_time [$__interval]
    )
)

  • What happened?
    No data error

  • What did you expect to happen?
    A panel with data

  • Can you copy/paste the configuration(s) that you are having problems with?

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    No data error

  • Did you follow any online instructions? If so, what is the URL?

I think the issue is with the regex, but I could be wrong. Can someone please provide a pointer? Thanks.