How to render gaps for null values?

charliefiskeaux · March 10, 2022, 1:21am

I’m doing some custom datasource plugin development, and the data is generally rendering fine, but sometimes there are null values which I want to render as gaps, but I can’t get this working. I can’t just include the null values because that throws errors when Grafana tries to render them. I found that data frame time fields apparently have a config property which can contain an interval property to handle this. Any values in my data which are not found at each time interval should be considered null and should render as gaps. The problem is that it doesn’t work…I can’t figure out how to cause gaps. Any help?

The data frame field interval config isn’t well documented, so I’m sort of stumbling around in the dark. I found source code references here:

github.com

grafana/grafana/blob/main/packages/grafana-data/src/types/dataFrame.ts#L74

      
        
            // Numeric Options
            unit?: string;
            decimals?: number | null; // Significant digits (for display)
            min?: number | null;
            max?: number | null;
            
            
// Interval indicates the expected regular step between values in the series.
            // When an interval exists, consumers can identify "missing" values when the expected value is not present.
            // The grafana timeseries visualization will render disconnected values when missing values are found it the time field.
            // The interval uses the same units as the values.  For time.Time, this is defined in milliseconds.
            interval?: number | null;
            
            
// Convert input values into a display string
            mappings?: ValueMapping[];
            
            
// Map numeric values to states
            thresholds?: ThresholdsConfig;
            
            
// Map values to a display color
            color?: FieldColor;

and an example datasource using it here:

github.com

grafana/grafana/blob/23956557d8c6a119b7de5be5c42024e29634d002/public/app/plugins/datasource/cloudwatch/datasource.ts#L512

      
        
              if (!dataframes || dataframes.length <= 0) {
                return { data: [] };
              }
            
            
  const lastError = findLast(res.results, (v) => !!v.error);
            
            
  dataframes.forEach((frame) => {
                frame.fields.forEach((field) => {
                  if (field.type === FieldType.time) {
                    // field.config.interval is populated in order for Grafana to fill in null values at frame intervals
                    field.config.interval = frame.meta?.custom?.period * 1000;
                  }
                });
              });
            
            
  return {
                data: dataframes,
                error: lastError ? { message: lastError.error } : null,
              };
            }),
            catchError((err) => {

But I can’t get it working. Here is an example of the data frames object I’m returning from my query method:

{
  "data": [
    {
      "refId": "A",
      "fields": [
        {
          "name": "Time",
          "type": "time",
          "config": {
            "interval": 60000
          },
          "values": {
            "buffer": [
              1645747380000,
              1645747440000,
              1645747500000,
              1645747560000,
              1645747620000,
              1645747680000,
              1645747740000,
              1645747800000,
              1645747860000,
              1645747920000,
              1645747980000,
              1645748040000,
              1645748100000,
              1645748160000,
              1645748220000,
              1645748280000,
              1645748340000,
              1645748400000,
              1645748460000,
              1645748520000,
              1645748580000,
              1645748640000,
              1645748700000,
              1645748760000,
              1645748820000,
              1645748880000,
              1645748940000,
              1645749000000,
              1645749060000,
              1645749120000,
              1645749180000,
              1645749240000,
              1645749300000,
              1645749360000,
              1645749420000,
              1645749480000,
              1645749540000,
              1645749600000,
              1645749660000,
              1645749720000,
              1645749780000,
              1645749840000,
              1645749900000,
              1645749960000,
              1645750020000,
              1645750080000,
              1645750140000,
              1645750200000,
              1645750260000,
              1645750320000,
              1645750380000,
              1645750440000,
              1645750500000,
              1645750560000,
              1645750620000,
              1645750680000,
              1645750740000,
              1645750800000,
              1645750860000,
              1645750920000,
              1645750980000,
              1645751040000,
              1645751640000,
              1645754400000,
              1645754520000,
              1645754580000,
              1645755840000,
              1645756020000,
              1645756980000,
              1645760340000,
              1645760820000,
              1645760880000,
              1645761060000,
              1645761180000,
              1645761540000,
              1645764960000,
              1645765020000,
              1645765260000,
              1645765320000,
              1645765440000,
              1645765500000,
              1645765620000,
              1645765680000,
              1645766100000,
              1645766400000,
              1645766460000,
              1645766520000,
              1645766640000,
              1645766760000,
              1645766940000,
              1645767960000,
              1645768740000,
              1645769580000,
              1645770240000,
              1645770480000,
              1645771500000,
              1645772100000
            ]
          }
        },
        {
          "name": "Value",
          "type": "number",
          "config": {
            "displayName": "median"
          },
          "values": {
            "buffer": [
              8.0078787878788,
              13.857142857143,
              7.0861111111111,
              7.0632653061224,
              7.0741935483871,
              7.0594713656388,
              7.0656862745098,
              7.0630541871921,
              7.0622317596567,
              8.0368421052632,
              9.0818181818182,
              8.0103448275862,
              7.0957983193277,
              7.07578125,
              8.0518518518519,
              9.0333333333333,
              7.0350318471338,
              7.0637037037037,
              7.040625,
              7.0237288135593,
              7.0651162790698,
              7.095,
              13.4,
              9.0666666666667,
              9.08,
              7.0767123287671,
              7.0611940298507,
              7.0794871794872,
              12.833333333333,
              8.03125,
              7.0823008849558,
              7.0617021276596,
              7.0306976744186,
              7.0333333333333,
              7.0495652173913,
              7.0626262626263,
              8.0478260869565,
              7.0710144927536,
              7.0616438356164,
              7.0702380952381,
              7.0623529411765,
              7.0429169157203,
              8.0759465478842,
              7.0515651678998,
              7.0634453781513,
              7.0826446280992,
              7.0879032258065,
              7.08,
              8.0333333333333,
              7.0793103448276,
              7.0668224299065,
              11.96,
              8.0192307692308,
              7.0768235294118,
              7.0839682539683,
              7.0539085545723,
              7.0373182552504,
              8.0631578947368,
              18.5,
              7.0435225618632,
              7.0281024819856,
              7.0219696969697,
              14.5,
              14.2,
              78.5,
              49.5,
              8.0666666666667,
              10.6,
              61.5,
              48.5,
              43.5,
              13.5,
              12.333333333333,
              9.05,
              12.5,
              45.5,
              12.25,
              10.5,
              28.5,
              89.5,
              52.333333333333,
              28.5,
              33.5,
              31.5,
              35.333333333333,
              9.05,
              44.5,
              46.5,
              32.5,
              11.5,
              29.5,
              11.076923076923,
              7.075,
              8.0666666666667,
              38.5,
              48.5,
              50.5
            ]
          },
          "labels": {
            "collector": "statsd",
            "source": "circonus-agent",
            "statsd_type": "timing"
          }
        }
      ],
      "target": "A"
    },
    {
      "refId": "A",
      "fields": [
        {
          "name": "Time",
          "type": "time",
          "config": {
            "interval": 60000
          },
          "values": {
            "buffer": [
              1645747380000,
              1645747440000,
              1645747500000,
              1645747560000,
              1645747620000,
              1645747680000,
              1645747740000,
              1645747800000,
              1645747860000,
              1645747920000,
              1645747980000,
              1645748040000,
              1645748100000,
              1645748160000,
              1645748220000,
              1645748280000,
              1645748340000,
              1645748400000,
              1645748460000,
              1645748520000,
              1645748580000,
              1645748640000,
              1645748700000,
              1645748760000,
              1645748820000,
              1645748880000,
              1645748940000,
              1645749000000,
              1645749060000,
              1645749120000,
              1645749180000,
              1645749240000,
              1645749300000,
              1645749360000,
              1645749420000,
              1645749480000,
              1645749540000,
              1645749600000,
              1645749660000,
              1645749720000,
              1645749780000,
              1645749840000,
              1645749900000,
              1645749960000,
              1645750020000,
              1645750080000,
              1645750140000,
              1645750200000,
              1645750260000,
              1645750320000,
              1645750380000,
              1645750440000,
              1645750500000,
              1645750560000,
              1645750620000,
              1645750680000,
              1645750740000,
              1645750800000,
              1645750860000,
              1645750920000,
              1645750980000,
              1645751040000,
              1645751640000,
              1645754400000,
              1645754520000,
              1645754580000,
              1645755840000,
              1645756020000,
              1645756980000,
              1645760340000,
              1645760820000,
              1645760880000,
              1645761060000,
              1645761180000,
              1645761540000,
              1645764960000,
              1645765020000,
              1645765260000,
              1645765320000,
              1645765440000,
              1645765500000,
              1645765620000,
              1645765680000,
              1645766100000,
              1645766400000,
              1645766460000,
              1645766520000,
              1645766640000,
              1645766760000,
              1645766940000,
              1645767960000,
              1645768740000,
              1645769580000,
              1645770240000,
              1645770480000,
              1645771500000,
              1645772100000
            ]
          }
        },
        {
          "name": "Value",
          "type": "number",
          "config": {
            "displayName": "p95"
          },
          "values": {
            "buffer": [
              71.375,
              174.44444444444,
              94.333333333333,
              113.33333333333,
              21.833333333333,
              43.5,
              96.333333333333,
              83.5,
              21.333333333333,
              175.71428571429,
              191.66666666667,
              173.33333333333,
              102.85714285714,
              96.333333333333,
              116.66666666667,
              117.5,
              9.02,
              93.5,
              11.666666666667,
              11.285714285714,
              92.5,
              146.66666666667,
              172,
              145,
              105.71428571429,
              103.33333333333,
              106.66666666667,
              107.5,
              136.66666666667,
              132.5,
              101.42857142857,
              99.666666666667,
              9.0166666666667,
              13.333333333333,
              82.5,
              106,
              183.33333333333,
              12.666666666667,
              71.666666666667,
              113.33333333333,
              92.333333333333,
              58.857142857143,
              103.75,
              31.375,
              15.5,
              102.5,
              87.5,
              57.5,
              96.666666666667,
              9.0833333333333,
              9.0555555555556,
              182.5,
              91.666666666667,
              93.714285714286,
              94.333333333333,
              17.714285714286,
              16.5,
              97.5,
              155,
              29.333333333333,
              11.75,
              8.0928571428571,
              185,
              178,
              185,
              186.66666666667,
              56.5,
              145,
              80.5,
              115,
              125,
              81.5,
              1850,
              27.5,
              115,
              125,
              1650,
              30.5,
              99.5,
              2050,
              2350,
              93.5,
              39.5,
              55.5,
              47.5,
              55.5,
              86.5,
              5050,
              59.333333333333,
              365,
              54.5,
              63.5,
              57.5,
              52.5,
              62.5,
              82.5,
              495
            ]
          },
          "labels": {
            "collector": "statsd",
            "source": "circonus-agent",
            "statsd_type": "timing"
          }
        }
      ],
      "target": "A"
    },
    {
      "refId": "A",
      "fields": [
        {
          "name": "Time",
          "type": "time",
          "config": {
            "interval": 60000
          },
          "values": {
            "buffer": [
              1645747380000,
              1645747440000,
              1645747500000,
              1645747560000,
              1645747620000,
              1645747680000,
              1645747740000,
              1645747800000,
              1645747860000,
              1645747920000,
              1645747980000,
              1645748040000,
              1645748100000,
              1645748160000,
              1645748220000,
              1645748280000,
              1645748340000,
              1645748400000,
              1645748460000,
              1645748520000,
              1645748580000,
              1645748640000,
              1645748700000,
              1645748760000,
              1645748820000,
              1645748880000,
              1645748940000,
              1645749000000,
              1645749060000,
              1645749120000,
              1645749180000,
              1645749240000,
              1645749300000,
              1645749360000,
              1645749420000,
              1645749480000,
              1645749540000,
              1645749600000,
              1645749660000,
              1645749720000,
              1645749780000,
              1645749840000,
              1645749900000,
              1645749960000,
              1645750020000,
              1645750080000,
              1645750140000,
              1645750200000,
              1645750260000,
              1645750320000,
              1645750380000,
              1645750440000,
              1645750500000,
              1645750560000,
              1645750620000,
              1645750680000,
              1645750740000,
              1645750800000,
              1645750860000,
              1645750920000,
              1645750980000,
              1645751040000,
              1645751640000,
              1645754400000,
              1645754520000,
              1645754580000,
              1645755840000,
              1645756020000,
              1645756980000,
              1645760340000,
              1645760820000,
              1645760880000,
              1645761060000,
              1645761180000,
              1645761540000,
              1645764960000,
              1645765020000,
              1645765260000,
              1645765320000,
              1645765440000,
              1645765500000,
              1645765620000,
              1645765680000,
              1645766100000,
              1645766400000,
              1645766460000,
              1645766520000,
              1645766640000,
              1645766760000,
              1645766940000,
              1645767960000,
              1645768740000,
              1645769580000,
              1645770240000,
              1645770480000,
              1645771500000,
              1645772100000
            ]
          }
        },
        {
          "name": "Value",
          "type": "number",
          "config": {
            "displayName": "p99"
          },
          "values": {
            "buffer": [
              87.5,
              265,
              177.5,
              203.33333333333,
              65.5,
              195,
              195,
              175,
              87.5,
              213.33333333333,
              315,
              298,
              173.33333333333,
              185,
              206.66666666667,
              233.33333333333,
              23.5,
              175,
              36.5,
              25.666666666667,
              165,
              195,
              236.66666666667,
              205,
              175,
              215,
              215,
              175,
              315,
              205,
              186.66666666667,
              175,
              23.5,
              93.5,
              126.66666666667,
              235,
              206.66666666667,
              75.5,
              405,
              225,
              186.15384615385,
              90.117647058824,
              195,
              81.111111111111,
              103.33333333333,
              176.66666666667,
              206.66666666667,
              107.5,
              173.33333333333,
              39.5,
              20.5,
              286,
              145,
              188.33333333333,
              176,
              87.666666666667,
              64.5,
              195,
              245,
              98.5,
              65.25,
              55.333333333333,
              185,
              2033.3333333333,
              185,
              186.66666666667,
              56.5,
              176.66666666667,
              80.5,
              115,
              165,
              81.5,
              1850,
              27.5,
              1950,
              355,
              1650,
              30.5,
              2150,
              2050,
              2350,
              93.5,
              39.5,
              55.5,
              47.5,
              55.5,
              86.5,
              5050,
              59.666666666667,
              365,
              54.5,
              79.5,
              57.5,
              52.5,
              62.5,
              82.5,
              495
            ]
          },
          "labels": {
            "collector": "statsd",
            "source": "circonus-agent",
            "statsd_type": "timing"
          }
        }
      ],
      "target": "A"
    },
    {
      "refId": "A",
      "fields": [
        {
          "name": "Time",
          "type": "time",
          "config": {
            "interval": 60000
          },
          "values": {
            "buffer": [
              1645747380000,
              1645747440000,
              1645747500000,
              1645747560000,
              1645747620000,
              1645747680000,
              1645747740000,
              1645747800000,
              1645747860000,
              1645747920000,
              1645747980000,
              1645748040000,
              1645748100000,
              1645748160000,
              1645748220000,
              1645748280000,
              1645748340000,
              1645748400000,
              1645748460000,
              1645748520000,
              1645748580000,
              1645748640000,
              1645748700000,
              1645748760000,
              1645748820000,
              1645748880000,
              1645748940000,
              1645749000000,
              1645749060000,
              1645749120000,
              1645749180000,
              1645749240000,
              1645749300000,
              1645749360000,
              1645749420000,
              1645749480000,
              1645749540000,
              1645749600000,
              1645749660000,
              1645749720000,
              1645749780000,
              1645749840000,
              1645749900000,
              1645749960000,
              1645750020000,
              1645750080000,
              1645750140000,
              1645750200000,
              1645750260000,
              1645750320000,
              1645750380000,
              1645750440000,
              1645750500000,
              1645750560000,
              1645750620000,
              1645750680000,
              1645750740000,
              1645750800000,
              1645750860000,
              1645750920000,
              1645750980000,
              1645751040000,
              1645751640000,
              1645754400000,
              1645754520000,
              1645754580000,
              1645755840000,
              1645756020000,
              1645756980000,
              1645760340000,
              1645760820000,
              1645760880000,
              1645761060000,
              1645761180000,
              1645761540000,
              1645764960000,
              1645765020000,
              1645765260000,
              1645765320000,
              1645765440000,
              1645765500000,
              1645765620000,
              1645765680000,
              1645766100000,
              1645766400000,
              1645766460000,
              1645766520000,
              1645766640000,
              1645766760000,
              1645766940000,
              1645767960000,
              1645768740000,
              1645769580000,
              1645770240000,
              1645770480000,
              1645771500000,
              1645772100000
            ]
          }
        },
        {
          "name": "Value",
          "type": "number",
          "config": {
            "displayName": "upper"
          },
          "values": {
            "buffer": [
              215,
              885,
              295,
              305,
              185,
              285,
              375,
              555,
              295,
              286.66666666667,
              625,
              386.66666666667,
              195,
              775,
              495,
              455,
              73.5,
              205,
              96.5,
              605,
              275,
              206.66666666667,
              365,
              245,
              255,
              285,
              275,
              5050,
              435,
              216.66666666667,
              285,
              245,
              135,
              215,
              255,
              275,
              535,
              295,
              815,
              275,
              495,
              905,
              2450,
              375,
              175,
              205,
              265,
              115,
              255,
              615,
              64.5,
              2250,
              365,
              296.66666666667,
              216.66666666667,
              275,
              425,
              215,
              395,
              305,
              176.66666666667,
              61.5,
              185,
              2550,
              185,
              186.66666666667,
              56.5,
              275,
              80.5,
              115,
              165,
              81.5,
              1850,
              27.5,
              1950,
              355,
              1650,
              30.5,
              2150,
              2050,
              2350,
              93.5,
              39.5,
              55.5,
              47.5,
              55.5,
              86.5,
              5050,
              59.666666666667,
              365,
              54.5,
              94.5,
              57.5,
              52.5,
              62.5,
              82.5,
              495
            ]
          },
          "labels": {
            "collector": "statsd",
            "source": "circonus-agent",
            "statsd_type": "timing"
          }
        }
      ],
      "target": "A"
    }
  ],
  "t": "ts"
}

marcusolsson · March 16, 2022, 4:13pm

I’m not familiar with the interval property, but I’d suspect that it’s up to the visualization to determine how to use the property, though I’m not sure what visualizations that makes use of it.

That being said, you should be able to set null values in your data frame. What errors are you getting?

Here’s an example using the Static data source:

null

charliefiskeaux · March 16, 2022, 4:35pm

Thank you! It turns out there was a bug elsewhere in the code I was trying to fix, which was masking things. Plain null values do work!

system · March 16, 2023, 4:35pm

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.