Transform data using Regular Expression

Hello,

First of all, I am using Grafana version 8.0.5. I have a set of data looking like that:

/v2/ports/PF-12345
/v2/services/cloud/PF-67891
/v2/services/cloud/connections/PF-123467

I am trying to modify this data set like below (basically removing v2 and the ID at the end of the URL.

ports
services/cloud
services/cloud/connections

I tried to use the Rename by regex under the Transform tab in my Panel but I am not having any success, I think it must be somewhere else.

Here is a regular expression I worked on regex101: build, test, and debug regex

Thanks for your help!
Romain

Welcome to the :grafana: community @romain!

I answered a similar post here that you might find helpful.

1 Like

Hello @melori.arellano,
I was able to upgrade Grafana to 8.5 and use the Value Mapping feature. Now I end up with a table cleaned up without any ID which is great.
Now, how can I do some post-processing on the result after value mapping?

For example, using Value Mapping, I end up with:

Port
Cloud Service
Cloud Service

I want to display a pie chart showing: 1 Port and 2 Cloud Services.
I tried to use Value Mapping and Select the Pie Chart but Grafana doesn’t seem to use the values post-value mapping but uses the original values.

See my value mapping settings:

And the result (which works great!):

Here I want to represent the number of “Cloud Router” and “Cloud Router BGP Settings” for example.

Thanks for your help,
Romain

What is the source of this data?
A database? A rest api? A file?

@yosiasz it is an elastic search database and the data are logs coming from GrayLogs (Elasticsearch)

1 Like

Elasticsearch has json document data. Please post one sample json document data.

Here is the sample data:

{
    "state": "Done",
    "series": [
      {
        "refId": "A",
        "fields": [
          {
            "config": {
              "filterable": true
            },
            "name": "timestamp",
            "type": "time",
            "values": [
              "2022-09-27 19:38:29.192",
              "2022-09-27 19:35:52.336",
              "2022-09-27 18:58:57.881",
              "2022-09-27 18:58:13.518",
              "2022-09-27 18:27:45.865",
              "2022-09-27 18:26:40.607",
              "2022-09-27 18:25:02.515",
              "2022-09-27 17:21:12.877",
              "2022-09-27 17:16:25.294",
              "2022-09-27 17:15:07.978",
              "2022-09-27 17:15:07.701",
              "2022-09-27 17:15:06.953",
              "2022-09-27 17:15:06.716",
              "2022-09-27 17:11:41.377",
              "2022-09-27 17:07:11.396",
              "2022-09-27 17:07:10.986",
              "2022-09-27 17:07:10.204",
              "2022-09-27 17:07:10.014",
              "2022-09-27 16:50:48.385",
              "2022-09-27 16:43:40.523",
              "2022-09-27 16:43:40.240",
              "2022-09-27 16:43:39.305",
              "2022-09-27 16:43:39.121",
              "2022-09-27 16:42:12.329",
              "2022-09-27 16:42:12.055",
              "2022-09-27 16:42:11.044",
              "2022-09-27 16:42:10.774",
              "2022-09-27 16:40:40.566",
              "2022-09-27 16:40:40.354",
              "2022-09-27 16:38:58.466",
              "2022-09-27 16:38:58.274"
            ],
            "state": null
          },
          {
            "config": {
              "filterable": true
            },
            "name": "path",
            "type": "string",
            "values": [
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567"
            ],
            "state": null
          }
        ],
        "first": [
          "2022-09-27 19:38:29.192",
          "2022-09-27 19:35:52.336",
          "2022-09-27 18:58:57.881",
          "2022-09-27 18:58:13.518",
          "2022-09-27 18:27:45.865",
          "2022-09-27 18:26:40.607",
          "2022-09-27 18:25:02.515",
          "2022-09-27 17:21:12.877",
          "2022-09-27 17:16:25.294",
          "2022-09-27 17:15:07.978",
          "2022-09-27 17:15:07.701",
          "2022-09-27 17:15:06.953",
          "2022-09-27 17:15:06.716",
          "2022-09-27 17:11:41.377",
          "2022-09-27 17:07:11.396",
          "2022-09-27 17:07:10.986",
          "2022-09-27 17:07:10.204",
          "2022-09-27 17:07:10.014",
          "2022-09-27 16:50:48.385",
          "2022-09-27 16:43:40.523",
          "2022-09-27 16:43:40.240",
          "2022-09-27 16:43:39.305",
          "2022-09-27 16:43:39.121",
          "2022-09-27 16:42:12.329",
          "2022-09-27 16:42:12.055",
          "2022-09-27 16:42:11.044",
          "2022-09-27 16:42:10.774",
          "2022-09-27 16:40:40.566",
          "2022-09-27 16:40:40.354",
          "2022-09-27 16:38:58.466",
          "2022-09-27 16:38:58.274"
        ],
        "parsers": {},
        "length": 31
      }
    ],
    "annotations": [],
    "request": {
      "app": "explore",
      "dashboardId": 0,
      "timezone": "browser",
      "startTime": 1664308073982,
      "interval": "30s",
      "intervalMs": 30000,
      "panelId": "1234",
      "targets": [
        {
          "refId": "A",
          "key": "1234",
          "query": "request_source: \"search\"\n",
          "alias": "",
          "metrics": [
            {
              "id": "1",
              "type": "raw_data",
              "settings": {
                "size": "500"
              }
            }
          ],
          "bucketAggs": [],
          "timeField": "timestamp",
          "datasource": {
            "type": "elasticsearch",
            "uid": "ABCD"
          }
        }
      ],
      "range": {
        "from": "2022-09-27T07:47:53.980Z",
        "to": "2022-09-27T19:47:53.980Z",
        "raw": {
          "from": "now-12h",
          "to": "now"
        }
      },
      "requestId": "explore_left",
      "rangeRaw": {
        "from": "now-12h",
        "to": "now"
      },
      "scopedVars": {
        "__interval": {
          "text": "30s",
          "value": "30s"
        },
        "__interval_ms": {
          "text": 30000,
          "value": 30000
        }
      },
      "maxDataPoints": 1261,
      "liveStreaming": false,
      "endTime": 1664308075444
    },
    "timeRange": {
      "from": "2022-09-27T07:47:53.980Z",
      "to": "2022-09-27T19:47:53.980Z",
      "raw": {
        "from": "now-12h",
        "to": "now"
      }
    },
    "timings": {
      "dataProcessingTime": 0.29999999701976776
    },
    "graphFrames": [],
    "tableFrames": [
      {
        "refId": "A",
        "fields": [
          {
            "config": {
              "filterable": true
            },
            "name": "timestamp",
            "type": "time",
            "values": [
              "2022-09-27 19:38:29.192",
              "2022-09-27 19:35:52.336",
              "2022-09-27 18:58:57.881",
              "2022-09-27 18:58:13.518",
              "2022-09-27 18:27:45.865",
              "2022-09-27 18:26:40.607",
              "2022-09-27 18:25:02.515",
              "2022-09-27 17:21:12.877",
              "2022-09-27 17:16:25.294",
              "2022-09-27 17:15:07.978",
              "2022-09-27 17:15:07.701",
              "2022-09-27 17:15:06.953",
              "2022-09-27 17:15:06.716",
              "2022-09-27 17:11:41.377",
              "2022-09-27 17:07:11.396",
              "2022-09-27 17:07:10.986",
              "2022-09-27 17:07:10.204",
              "2022-09-27 17:07:10.014",
              "2022-09-27 16:50:48.385",
              "2022-09-27 16:43:40.523",
              "2022-09-27 16:43:40.240",
              "2022-09-27 16:43:39.305",
              "2022-09-27 16:43:39.121",
              "2022-09-27 16:42:12.329",
              "2022-09-27 16:42:12.055",
              "2022-09-27 16:42:11.044",
              "2022-09-27 16:42:10.774",
              "2022-09-27 16:40:40.566",
              "2022-09-27 16:40:40.354",
              "2022-09-27 16:38:58.466",
              "2022-09-27 16:38:58.274"
            ],
            "state": null
          },
          {
            "config": {
              "filterable": true
            },
            "name": "_id",
            "type": "string",
            "values": [
              "f65c4ad4-3e9b-123456",
              "98d7dcd1-3e9b-123456",
              "70ecb152-3e96-123456",
              "567be4d3-3e96-123456",
              "151d9140-3e92-123456",
              "ee37d593-3e91-123456",
              "b3c0cb14-3e91-123456",
              "c91bcc76-3e88-123456",
              "1db244a5-3e88-123456",
              "ef9c2c13-3e87-123456",
              "ef72d224-3e87-123456",
              "ef003a39-3e87-123456",
              "eede0c30-3e87-123456",
              "74789501-3e87-123456",
              "d38df8bf-3e86-123456",
              "d34dbb6a-3e86-123456",
              "d2d7a100-3e86-123456",
              "d2b9dfd0-3e86-123456",
              "89a0b7d1-3e84-123456",
              "8a9a4f89-3e83-123456",
              "8a6f9607-3e83-123456",
              "89dfd920-3e83-123456",
              "89c37780-3e83-123456",
              "56096e93-3e83-123456",
              "55dfc685-3e83-123456",
              "554449d1-3e83-123456",
              "551bb332-3e83-123456",
              "1f5893d4-3e83-123456",
              "1f3665d1-3e83-123456",
              "e27d0c83-3e82-123456",
              "e25dc4ac-3e82-123456"
            ],
            "state": null
          },
          {
            "config": {
              "filterable": true
            },
            "name": "path",
            "type": "string",
            "values": [
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
              "/v2/services/cloud-routers/PF-L3-CUST-1234567"
            ],
            "state": null
          }
        ],
        "first": [
          "2022-09-27 19:38:29.192",
          "2022-09-27 19:35:52.336",
          "2022-09-27 18:58:57.881",
          "2022-09-27 18:58:13.518",
          "2022-09-27 18:27:45.865",
          "2022-09-27 18:26:40.607",
          "2022-09-27 18:25:02.515",
          "2022-09-27 17:21:12.877",
          "2022-09-27 17:16:25.294",
          "2022-09-27 17:15:07.978",
          "2022-09-27 17:15:07.701",
          "2022-09-27 17:15:06.953",
          "2022-09-27 17:15:06.716",
          "2022-09-27 17:11:41.377",
          "2022-09-27 17:07:11.396",
          "2022-09-27 17:07:10.986",
          "2022-09-27 17:07:10.204",
          "2022-09-27 17:07:10.014",
          "2022-09-27 16:50:48.385",
          "2022-09-27 16:43:40.523",
          "2022-09-27 16:43:40.240",
          "2022-09-27 16:43:39.305",
          "2022-09-27 16:43:39.121",
          "2022-09-27 16:42:12.329",
          "2022-09-27 16:42:12.055",
          "2022-09-27 16:42:11.044",
          "2022-09-27 16:42:10.774",
          "2022-09-27 16:40:40.566",
          "2022-09-27 16:40:40.354",
          "2022-09-27 16:38:58.466",
          "2022-09-27 16:38:58.274"
        ],
        "parsers": {},
        "length": 31
      }
    ],
    "logsFrames": [],
    "traceFrames": [],
    "nodeGraphFrames": [],
    "graphResult": null,
    "tableResult": {
      "refId": "A",
      "fields": [
        {
          "config": {
            "filterable": true
          },
          "name": "timestamp",
          "type": "time",
          "values": [
            "2022-09-27 19:38:29.192",
            "2022-09-27 19:35:52.336",
            "2022-09-27 18:58:57.881",
            "2022-09-27 18:58:13.518",
            "2022-09-27 18:27:45.865",
            "2022-09-27 18:26:40.607",
            "2022-09-27 18:25:02.515",
            "2022-09-27 17:21:12.877",
            "2022-09-27 17:16:25.294",
            "2022-09-27 17:15:07.978",
            "2022-09-27 17:15:07.701",
            "2022-09-27 17:15:06.953",
            "2022-09-27 17:15:06.716",
            "2022-09-27 17:11:41.377",
            "2022-09-27 17:07:11.396",
            "2022-09-27 17:07:10.986",
            "2022-09-27 17:07:10.204",
            "2022-09-27 17:07:10.014",
            "2022-09-27 16:50:48.385",
            "2022-09-27 16:43:40.523",
            "2022-09-27 16:43:40.240",
            "2022-09-27 16:43:39.305",
            "2022-09-27 16:43:39.121",
            "2022-09-27 16:42:12.329",
            "2022-09-27 16:42:12.055",
            "2022-09-27 16:42:11.044",
            "2022-09-27 16:42:10.774",
            "2022-09-27 16:40:40.566",
            "2022-09-27 16:40:40.354",
            "2022-09-27 16:38:58.466",
            "2022-09-27 16:38:58.274"
          ],
          "state": null
        },
        {
          "config": {
            "filterable": true
          },
          "name": "_id",
          "type": "string",
          "values": [
            "f65c4ad4-3e9b-123456",
            "98d7dcd1-3e9b-123456",
            "70ecb152-3e96-123456",
            "567be4d3-3e96-123456",
            "151d9140-3e92-123456",
            "ee37d593-3e91-123456",
            "b3c0cb14-3e91-123456",
            "c91bcc76-3e88-123456",
            "1db244a5-3e88-123456",
            "ef9c2c13-3e87-123456",
            "ef72d224-3e87-123456",
            "ef003a39-3e87-123456",
            "eede0c30-3e87-123456",
            "74789501-3e87-123456",
            "d38df8bf-3e86-123456",
            "d34dbb6a-3e86-123456",
            "d2d7a100-3e86-123456",
            "d2b9dfd0-3e86-123456",
            "89a0b7d1-3e84-123456",
            "8a9a4f89-3e83-123456",
            "8a6f9607-3e83-123456",
            "89dfd920-3e83-123456",
            "89c37780-3e83-123456",
            "56096e93-3e83-123456",
            "55dfc685-3e83-123456",
            "554449d1-3e83-123456",
            "551bb332-3e83-123456",
            "1f5893d4-3e83-123456",
            "1f3665d1-3e83-123456",
            "e27d0c83-3e82-123456",
            "e25dc4ac-3e82-123456"
          ],
          "state": null
        },
        {
          "config": {
            "filterable": true
          },
          "name": "path",
          "type": "string",
          "values": [
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/aws",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
            "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
            "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567",
            "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/bgp-settings/a12334a1-a12b-1234-c12d-abc123456cdf/prefixes",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections/PF-L3-CON-1234567/bgp/a12334a1-a12b-1234-c12d-abc123456cdf",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567/connections",
            "/v2/services/cloud-routers/PF-L3-CUST-1234567"
          ],
          "state": null
        }
      ],
      "first": [
        "2022-09-27 19:38:29.192",
        "2022-09-27 19:35:52.336",
        "2022-09-27 18:58:57.881",
        "2022-09-27 18:58:13.518",
        "2022-09-27 18:27:45.865",
        "2022-09-27 18:26:40.607",
        "2022-09-27 18:25:02.515",
        "2022-09-27 17:21:12.877",
        "2022-09-27 17:16:25.294",
        "2022-09-27 17:15:07.978",
        "2022-09-27 17:15:07.701",
        "2022-09-27 17:15:06.953",
        "2022-09-27 17:15:06.716",
        "2022-09-27 17:11:41.377",
        "2022-09-27 17:07:11.396",
        "2022-09-27 17:07:10.986",
        "2022-09-27 17:07:10.204",
        "2022-09-27 17:07:10.014",
        "2022-09-27 16:50:48.385",
        "2022-09-27 16:43:40.523",
        "2022-09-27 16:43:40.240",
        "2022-09-27 16:43:39.305",
        "2022-09-27 16:43:39.121",
        "2022-09-27 16:42:12.329",
        "2022-09-27 16:42:12.055",
        "2022-09-27 16:42:11.044",
        "2022-09-27 16:42:10.774",
        "2022-09-27 16:40:40.566",
        "2022-09-27 16:40:40.354",
        "2022-09-27 16:38:58.466",
        "2022-09-27 16:38:58.274"
      ],
      "parsers": {},
      "length": 31
    },
    "logsResult": null
  }

The goal is to show a Pie Chart with the 34 Cloud Routers (cloud-routers) and 4 BGP Settings (bgp-settings).

1 Like